Apache Airflow is an open-source tool for orchestrating complex computational workflows and data processing pipelines. If you find yourself running cron task which execute ever longer scripts, or keeping a calendar of big data processing batch jobs then Airflow can probably help you. This article provides an introductory tutorial for people who want to get started writing pipelines with Airflow.
This article outlines steps needed to create a REST API using Flask and Flask-RESTPlus. These tools combine into a framework, which automates common tasks:
- API input validation
- formatting output (as JSON)
- generating interactive documentation (with Swagger UI)
- turning Python exceptions into machine-readable HTTP responses
Docker is an open-source software container management system. It allows you to create an isolated, self-contained environment to run your application. In this article we will walk thought steps needed to create a Docker image containing a Django application and demonstrate how to run it and access the container.
Slides for a presentation I gave last week at DevCon in Warsaw.
Celery is a powerful, production-ready asynchronous job queue, which allows you to run time-consuming Python functions in the background. A Celery powered application can respond to user requests quickly, while long-running tasks are passed onto the queue. In this article we will demonstrate how to add Celery to a Django application using Redis.
Jenkins is an easy-to-use open-source continuous integration server. In this post we’ll go through steps needed to set up Jenkins to deploy your Django application and run unit tests whenever someone commits code to your project’s repository. If the new code causes any of your tests to fail, Jenkins will send the commiter an email alert.
I recently wrote a book about administering Unix and Linux servers with the aid of the friendly and versatile utility called Webmin. The book covers a wide variety of topics, from setting up and securing a new server to running and monitoring databases (MySQL, Postgres), a web server (Apache) and a mail server (Postfix).
The book is written as a series of 120 step-by-step recipes, which should be easily accessible to both novice and professional system administrators. Check it out to see how Webmin can make your life easier. The book is currently available from Packt Publishing.
Performance testing web applications is a little tricky when we want to test features accessible only to logged in users. Automated test recorders intercept session cookies and hardcode them into test scenarios. When we run these tests later, different session IDs are generated and recorded cookie values don’t grant access anymore. This text demonstrates how to write a scenario for The Grinder to test a Django application. Tests include user login, submitting CSRF-protected forms and AJAX requests.
Nginx makes a great server for your Gunicorn-powered Django applications. In this article I will demonstrate how you can run multiple Django applications on the same Nginx server, hosting sites on two different domains. Each application will be set up in its own Virtualenv and each will be owned by and run as a different user to limit consequences of a potential security breach.