未加星标

Microservices in Flask

字体大小 | |
[开发(python) 所属分类 开发(python) | 发布者 店小二05 | 时间 2016 | 作者 红领巾 ] 0人收藏点击收藏

I recently gave a talk about microservices in Flask on Wrocpy . This blog post is a translation of that talk into English.

Table of Contents:

Monolithic application

Microservices talk usually starts with a monolithic application. In my case, it is the same. I work on a project where I have a big monolithic application. If I wanted to take some part of it and make some microservice from for instance django app it would be impossible. There are too many internal imports from every part of application:

from app.users.models import UserSiteProfile from app.utils.cache import cache_key_user from app.sites.models import SiteProfile from app.sites.utils import site_terms from app.utils.users import get_homepage_url from app.utils.views import ThemedFormView, ThemedTemplateView from app.authentication import signals from app.authentication.forms import AuthForm, EmailForm from app.authentication.idp import ( is_valid_idp, MultipleIDPError, saml_available, site_idp, user_idp_lookup) from app.authentication.loginchecks import ( check_account_expiration, get_site_login_errors, pre_login_checks ) from app.saml.utils import site_specific_config_loader from app.saml.views import _get_subject_id

Snippet from above presents exemplary imports of random python module in my project.

So where are these microservices? In my team, we decided to have new parts of the application made as a microservices. So right now from the architectural point of view I have a big monolithic application and small microservices that are around this big thing, like in this picture:


Microservices in Flask

Picture from link .

We choose the flask as a tool that will be used in our microservices. It doesn't set any boundaries and it's flexible but don't have batteries included . Let's start with the first flask extension that helps us building microservices:

Django has a set of powerful commands available. To enable such a feature in flask you needFlask-Script. It allows you to create commands such as runserver or shell . In listing below I created a runserver command:

from flask.ext.script import Manager, Shell, Server from my_app.application import app manager = Manager(app) manager.add_command( 'runserver', Server(host='0.0.0.0', port=7000, use_debugger=True) )

To communicate between microservices I use REST. To ease yourself when creating resources and endpoints you can useFlask-RESTful. It is superuseful when you need to create REST API. It is easy- you specify endpoint with resource and rest is done by Flask-RESTful. It also has request parsing and it is very easy to create other representations like xml. The snippet below shows it:

from flask_restful import Resource class MyResource(Resource): def get(): return {'OK'} @api.representation('application/xml') def output_xml(data, code, headers=None): response = make_response(dicttoxml.dicttoxml(data), code) inject_headers(response, headers) return response

Marshallow & flask-marshallow

To serialize or deserialize objects you can use flask-marshallow . In code below, I created a simple Schema with id, name and url. Then when the user enters /api/custom/1 I can easily serialize objects from a database and return JSON.

ma = Marshmallow(app) class CustomSchema(ma.Schema): id = ma.Int(dump_only=True) name = ma.Str(required=True) url = ma.Url(relative=True) @app.route('/api/custom/<id>') def users(): all_obj = Object.all() result = object_schema.dump(all_obj) return jsonify(result.data)

Most of the modern frameworks have support for ORM- the same is with Flask. There is an extension called Flask-SQLAchemy that adds support for SQLAlchemy . Take this snippet for instance:

from flask.ext.sqlalchemy import SQLAlchemy db = SQLAlchemy(app) class MyModel(db.Model): GROUP = 'group' USER = 'user' TYPES = [ (GROUP, GROUP), (USER, USER), ] __tablename__ = 'my_model' id = db.Column(db.Integer, primary_key=True) model_type = db.Column(ChoiceType(TYPES)) model_value_id = db.Column(db.Integer, db.ForeignKey('model_value.id')) value = db.Column(db.String(1024)) def __init__(self, model_type, model_value_id, value): self.share_type = share_type self.rule_value_id = rule_value_id self.value = value

I created MyModel class that will be translated to the table in a database. I also add columns to that table like model_type , model_value_id or value .

Flask SQLAlchemy is layer sitting on top of SQLAlchemy so you can use all advantages of ORM like having queries written in python.

When your database keeps getting larger there is a need for database migrations to make sure that everyone has the same database structure. To accomplish that we useFlask-Migrate. It is using Alembic under the hood so you have to make sure that adjust your migration files after generation. Example migration file can look as follows:

def upgrade(): op.create_table( 'my_model', sa.Column('id', sa.Integer(), nullable=False), sa.Column( 'model_type', sqlalchemy_utils.types.choice.ChoiceType(TYPES), nullable=True ), sa.PrimaryKeyConstraint('id'), sa.Column('value', sa.String(length=1024), nullable=True), ) op.create_table( 'my_model_values', sa.Column('id', sa.Integer(), nullable=False), sa.Column('model_value_id', sa.Integer(), nullable=True), sa.Column('value', sa.String(length=1024), nullable=True), sa.ForeignKeyConstraint(['model_value_id'], ['my_model.id']), sa.PrimaryKeyConstraint('id') )

In snipped above I created two tables: my_model and my_model_values with respective columns. Also my_model_values has ForeignKey relation to my_model by their ids.

During the development of microservices, we write unit tests as well as integration ones. Testing how well microservices behave with each other can be tricky: we mock whole external services. Because of that, we need to keep them up to date with real microservices. Nature of this system causes some difficulties while an error occurs: I got an error from external microservice in most cases with a form of HTTP status code and a small message in JSON or XML. Then I need to debug not only my microservice but also external one.

After testing is done we deploy microservice using few tools:

We use puppet for managing and provisioning our microservices. Especially we use an R10k module for puppet: gtihub link .

To make sure that every microservice has the same structure we also use cookiecutter . Thanks to that puppet knows that config file is always in this location or there will be logs stored there. Example microservice structure will look as follows:

└── flask_microservice ├── AUTHORS.rst ├── debian ├── docs │ ├── make.bat │ ├── Makefile │ └── source │ ├── authors.rst │ ├── conf.py │ ├── contributing.rst │ ├── history.rst │ ├── index.rst │ ├── readme.rst │ ├── technical_details.rst │ └── usage.rst |── HISTORY.rst ├── MANIFEST.in ├── README.rst ├── requirements.txt ├── setup.cfg ├── setup.py ├── src │ ├── flask_microservice │ │ ├── application.py │ │ ├── default_config.ini │ │ ├── __init__.py │ │ └── manage.py │ └── tests │ ├── conftest.py │ └── test_flask_microservice.py └── tox.ini

To distribute packages we usedh-virtualenv. This basically is taking python virtual enviroments and packing it to deb packages. So to have new code released we just run jenkins job to create new deb.

That's all for today! The slides from this presentation can be found here: presentation . Do you also use flask to build microservices? Please give your comments about that.

Special thanks to Kasia for being editor for this post. Thank you.

Cover image by NASA JPL - Nasa JPL under CC0 Public Domain .

本文开发(python)相关术语:python基础教程 python多线程 web开发工程师 软件开发工程师 软件开发流程

主题: SQLRESTDjangoXMLAUUTAUTRYADMNASA
分页:12
转载请注明
本文标题:Microservices in Flask
本站链接:http://www.codesec.net/view/480685.html
分享请点击:


1.凡CodeSecTeam转载的文章,均出自其它媒体或其他官网介绍,目的在于传递更多的信息,并不代表本站赞同其观点和其真实性负责;
2.转载的文章仅代表原创作者观点,与本站无关。其原创性以及文中陈述文字和内容未经本站证实,本站对该文以及其中全部或者部分内容、文字的真实性、完整性、及时性,不作出任何保证或承若;
3.如本站转载稿涉及版权等问题,请作者及时联系本站,我们会及时处理。
登录后可拥有收藏文章、关注作者等权限...
技术大类 技术大类 | 开发(python) | 评论(0) | 阅读(53)