Automate ML Pipelines Using Apache Airflow

Course Cover
compare button icon

Course Features

icon

Duration

45 minutes

icon

Delivery Method

Online

icon

Available on

Limited Access

icon

Accessibility

Mobile, Desktop, Laptop

icon

Language

English

icon

Subtitles

English

icon

Level

Intermediate

icon

Teaching Type

Self Paced

icon

Video Content

45 minutes

Course Description

If you master Apache Airflow you will gain practical experience creating an KNN classifier to be used in Iris. Iris dataset, employing Apache Airflow for workflow automation. Additionally, you will have learned ways to implement the learned model to predict and also how to create an DAG ( Directed Acyclic graph ) to be used in data pipelines. This will improve productivity as well as reduce costs. It will also provide faster time-to-insight. These are skills that are vital for anyone who is who works on classification tasks as well as data pipelines, and are applicable to broad variety of other datasets as well as workflows.This guided project is a combination of two of the most powerful technology: Apache Airflow and machine learning algorithms for classification. In this course, you'll be taught how to use Airflow to build an automated workflow that trains and evaluates a classification model on the data. It will also cover different algorithms for classification, like KNN ( K-Nearest Neighbors ) which one is the most suitable for the data. At the end of the course, you'll have a complete process for developing and testing a classification model, which makes it simple to put the model into production and create the DAG (Directed Acyclic Graph)The Iris dataset is a popular and extensively used data set in the machine learning field. It is comprised of measurements of three iris species flowers, and is widely utilized as a benchmark data set for models of classification. In this course you will get practical experience creating a classification model that utilizes the K-Nearest Neighbors ( KNN ) algorithm, that is a well-known machine-learning algorithm used for classification tasks.This project offers a well-structured method for creating an effective classification model that can easily be adapted to other workflows and datasets. The utilization in Apache Airflow allows for the automated process of the entire procedure starting with data preparation to the evaluation of models and their deployment, making it simple to integrate this process into your projects.It can be a great opportunity to gain experience and master the use of Apache Airflow, an open-source platform that allows you to programmatically create schedulers, monitoring, and workflows. Airflow offers a user-friendly interface to build and testing pipelines for data, making it a vital tool for anyone working with data scientists or engineers.

Course Overview

projects-img

International Faculty

projects-img

Post Course Interactions

projects-img

Instructor-Moderated Discussions

projects-img

Case Studies, Captstone Projects

Skills You Will Gain

Prerequisites/Requirements

You will also need some prior experience working with Python to understand code easily.

To complete this guided project, you will need a basic understanding of Machine Learning.

What You Will Learn

Implement an Apache Airflow workflow to automate the process of data preprocessing, model training, and evaluation.

Learn how to create a DAG using Apache Airflow, which is a collection of tasks and dependencies that represent a data pipeline.

Understand the K-Nearest Neighbors algorithm and its use in classification tasks.

Using Airflow to schedule and monitor the execution of the workflow, and to visualise the results.

Course Instructors

Jigisha Barbhaya

Data Scientist

I am a Data scientist at IBM and Lead instructor at Skills network. I love to learn and educate. I have completed my MSc(Computer Application) specialisation in Data science from Symbiosis University.

Joseph Santarcangelo

PhD., Data Scientist

Joseph Santarcangelo is currently working as a Data Scientist at IBM. Joseph has a Ph.D. in Electrical Engineering. His research focused on using machine learning, signal processing, and computer vision to determine how videos impact human cognition.
Course Cover