Kafka is a high throughput distributed messaging system. In this exercise, we’ll implement a simple Kafka service to read a feed from Twitter and write it to Elastic Search, using Python.

Kafka Basics.

Pre-requisites:

  • Kafka Server installed locally.
  • Twitter developer account.
  • Bonsai Elastic Search (or local installation of Elastic Search)

Install Kafka.

  1. Download and extract:

2. Go into the extracted folder and launch Zookeeper and Kafka Server:

3. Create a topic, which is simply a particular stream of data. In this example, the topic is “bitcoin”


The six principles of Kanban

  • Visualize the workflow
  • Limit work in progress [WIP]
  • Manage flow
  • Make policies explicit
  • Implement feedback loops
  • Improve collaboratively, evolve experimentally

The seven principles of Lean

  • Eliminate waste: maximize value
  • Amplify learning: via frequent communication and feedback
  • Decide as late as possible: Early planning, Late decision making.
  • Deliver as Fast as possible: Maximize ROI
  • Empower Team: self-managed and trusted.
  • Build Quality In: Quality Assurance rather than Quality Control
  • See the whole: See the system, not the parts of a system

The seven wastes of Lean Software Development

  • Partially done work
  • extra features
  • relearning: Lack of documentation, poor coding practices, and consistent task switching between team members all create chaos as each one attempts…

Agile in a nutshell.

Agile is a mindset that allows an organization to focus on better ways to produce value for the customer and practice continuous improvement.

Agile Manifesto.

Individuals and Interactions over processes and tools.

Working software over comprehensive documentation.

Customer collaboration over contract negotiation.

Responding to change over following a plan.

Agile Principles.

Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.

Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.

Deliver working software frequently, from a couple of weeks to a couple of months. with a preference to the shorter timescale.


No matter how big or difficult your goal seems to be, there is always a way to get there. Here I share some thoughts that you can apply to your own circumstances to reach whatever goal you have defined for yourself.

Scope.
Clear definition of what IS and, most importantly, what IS NOT your goal. Remove all the clutter. Define exactly what you want to achieve and when in a single sentence. If your goal is too vague, you will end up wasting a lot of time doing things that do not contribute to your goal. Keep it simple, clear…

Jorge Contreras

I develop software professionally, and take aerial photography and race triathlons for fun.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store