A Kafka Broker is nothing more, nothing less, than a JVM process that runs on a machine that serves data store/fetch requests from clients, i.e Producers and Consumers.

The data itself is stored in specific directories configured with log.dirs, typically a list of disk mount points of that Broker. You…


Since Spark executes an application in a distributed fashion, it is impossible to atomically write the result of the job. For example, when you write a Dataframe, the result of the operation will be a directory with multiple files in it, one per Dataframe's partition (e.g part-00001-...). …


On the 30th of May, XTech Community promoted a hands-on “Spark Intro & Beyond” meetup. …

Andriy Zabolotnyy

Software & Data Engineer @ tb.lx by Daimler Trucks & Buses

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store