Intro to Building Data Pipelines in Python with Luigi

01:45 PM - 02:10 PM on August 16, 2015, Room 702

Adrian Cruz

Audience level:

novice

Watch:

http://youtu.be/ymF2R_tY1f8

Description

Luigi is a Python package open sourced by the folks at Spotify. It facilitates building out the workflows of working with data. You will learn about data pipelines and why Luigi helps out dramatically. This will be an introductory level talk, but all experience levels are welcome.

Abstract

I will discuss the other options outside of Luigi that you can use and explore the way tasks were handled prior to any of these tools existing. What you will learn are the main concepts of Luigi in order to be able to start using it and the benefits of having a tool like Luigi.