Kafka – Docker compose example for Kafka

Kafka docker compose example. Please set value of KAFKA_ADVERTISED_HOST_NAME for local host IP address.

version: '2.1'
  
services: 
  zookeeper:
    image: wurstmeister/zookeeper:3.4.6
    ports:
      - "2181:2181"

  kafka:
    image: wurstmeister/kafka:2.12-2.2.0
    ports:
      - '9092:9092'
    tmpfs:
      - /kafka/:rw
    depends_on:
      - zookeeper
    environment:
      KAFKA_ADVERTISED_HOST_NAME: [HOST_IPADDRESS]
      KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181'
Advertisements
Posted in kafka, Programming | Leave a comment

Time – days of clock vs Monotonic clocks

Getting days of clocks
– Linux: clock_gettime(CLOCK_REALTIME)
– Java: System.currentTimeMillis()

Getting monotonic clocks
– Linux: clock_gettime(CLOCK_MONOTONIC)
– Java: System.nanoTime()

It’s important to know difference between days of clocks and monotonic clocks because days of clocks is measured from a certain epock(Java is using 1970 Jan 1 in UTC) it can be backward when time is adjusted by NTP.
But monotonic clocks will never go back, it will guarantee to be always forward so if application want to measure the duration of time, monotonic clocks should be used.
However it’s meaningless on comparing monotonic clocks between two machines.

Posted in Programming | Leave a comment

Database – Partitioning name in various database

Various database called partitioning in various name. It’s interesting but they are all same.

HBase: region
Cassandra: vnode
MongoDB: shard
Elasticsearch: shard
Bigtable: tablet
Riak: vnode
Couchbase: vBucket

Posted in Programming | Leave a comment

Reactive Stream – Example of implementation of Publisher

View at Medium.com

Posted in Programming | Leave a comment

Reactive Programming

It is worth to read through The Reactive Manifesto definition.

https://www.reactivemanifesto.org/glossary

Posted in Programming | Leave a comment

Linux – Network settings for high traffic generating application

/etc/sysctl.conf

CENT OS example

# sysctl -w net.core.wmem_default=262144

# default socket memory buffer per a socker
net.core.wmem_default=131072(128KB)
net.core.rmem_default=131072(128KB)

# max socker memory buffer per a socker
net.core.wmem_max=2097152(2MB)
net.core.rmem_max=2097152(2MB)

# tcp buffer size (min, default, max) 4KB, 64KB, 2MB
net.ipv4.tcp_wmem=4096 65536 2048000

# enable TCP window scaling: client can transfer data more efficiently, it will be buffered in server side.
net.ipv4.tcp_window_scaling=1

# allow to accept simultaneous connections
net.ipv4.tcp_max_syn_backlog=1024

Posted in Linux, Programming | Leave a comment

Websocket protocol details

Websocket protocol details.
http://lucumr.pocoo.org/2012/9/24/websockets-101/

Posted in Programming, Web | Leave a comment