- 1. RESTful Web API With Python, Flask and Mongo
- 8. gestionaleamica.com invoicing & accounting
- 10. 进入 Python Flask 和 Mongo 的学习
- 11. 那么 REST 都是关于什么的?
- 12. REST 不是一个标准,也不是一个协议
- 14. REST 是一个架构风格的网络应用程序
- 15. REST 松散地定义了一组简单的规则以及大多数API的实现
- 16. #1 资源来源的具体信息, 一个web页面而不是资源准确的说它是资源的一种表现形式
- 18. #2 全球的每个资源永久标识是唯一标识 (想一想一个HTTP URI)
- 19. #3 标准接口用来交换资源的表示 (think the HTTP protocol)
- 20. #4 一组包括了关注点,无状态,缓存能力,分层系统,统一接口的分离,稍后将会讲到
- 21. World Wide Web 是建立在 REST 之上(人们使用)
- 22. RESTful Web APIs 也是建立在 REST(计算机使用)
- 23. 可以看看我是怎样向老婆描述REST的: Ryan Tomayko http://tomayko.com/writings/rest-to-my-wife
- 24. The Real Stuff Representational State Transfer (REST) by Roy Thomas Fielding http://www.ics.uci.edu/~fielding/pubs/dissertation/rest_arch_style.ht
- 25. RESTful Web API Nuts & Bolts
- 26. Flask & Mongo
- 27. 让我们一点一滴地来看Flask 网络开发
- 28. 高雅简单,内置开发服务器和调试器,有明确的尚可执行的对象,100%符合WSGI规则,响应对象是WSGI应用程序本身
from flask import Flask app = Flask(__name__) @app.route("/") def hello(): return "Hello World!"
if __name__ == "__main__": app.run(debug=True) - 29. RESTful 请求调度
@app.route(/user/<username>)
def show_user_profile(username):
return User %s % username @app.route(/post/<int:post_id>)
def show_post(post_id):
return Post %d % post_id - 33.最小的Footprint只有800行源代码
- 34. Heavily Tested 1500 lines of tests
- 35. Unittesting Support one day I will make good use of it
- 36. Bring Your Own Batteries we aim for flexibility
- 37. No built-in ORM we want to be as close to the bare metal as possible
- 38. No form validation we don’t need no freaking form validation
- 39. No data validation Python offers great tools to manipulate JSON, we can tinker something ourselves
- 40. Layered API built on Werkzeug, Jinja2, WSGI
- 41. Built by the Pros The Pocoo Team did Werkzeug, Jinja2, Sphinx, Pygments, and much more
- 42. Excellent Documentation Over 200 pages, lots of examples and howtos
- 43. Active Community Widely adopted, extensions for everything
- 44. “Flask is a sharp tool for building sharp services” Kenneth Reitz, DjangoCon 2012
- 45. MongoDB scalable, high-performance, open source NoSQL database
- 46. Similarity with RDBMS made NoSQL easy to grasp (even for a dumbhead like me)
- 47. Terminology RDBMS Mongo Database Database Table Collection Rows(s) JSON Document Index Index Join Embedding & Linking
- 48. JSON-style data store true selling point for me
- 49. JSON & RESTful API GET Client Mongo JSON JSON accepted media type (BSON) maybe we can push directly to client?
- 50. JSON & RESTful API GET Client API Mongo JSON JSON/dict JSON accepted media type maps to python dict (BSON) almost.
- 51. JSON & RESTful API POST Client API Mongo JSON JSON/dict JSON maps to python dict objects (BSON) (validation layer) also works when posting (adding) items to the database
- 52. What about Queries? Queries in MongoDB are represented as JSON-style objects //
select * from things where x=3 and y="foo" db.things.find({x: 3, y: "foo”});
- 53. JSON & RESTful API FILTERING & SORTING ?where={x: 3, y: "foo”} Client API Mongo (very) thin native parsing JSON Mongo & validation (BSON) query syntax layer
- 54. JSON all along the pipeline mapping to and from the database feels more natural
- 55. Schema-less dynamic objects allow for a painless evolution of our schema (because yes, a schema exists at any point in time)
- 56. ORM Where we’re going we don’t need ORMs.
- 57. PyMongo official Python driver all we need to interact with the database
- 58. Also in MongoDB • setup is a breeze • lightweight • fast inserts, updates and queries • excellent documentation • great support by 10gen • great community
- 59. A Great Introduction To MongoDB The Little MongoDB Book by Karl Seguin http://openmymind.net/2011/3/28/The-Little-MongoDB-Book/
- 60. Shameless Plug Il Piccolo Libro di MongoDB by Karl Seguin, traduzione di Nicola Iarocci http://nicolaiarocci.com/il-piccolo-libro-di-mongodb-edizione-italiana/
- 61. MongoDB Interactive Tutorial http://tutorial.mongly.com/tutorial/index
- 62. RESTful Web APIs are really just collection of resources accesible through to a uniform interface
- 63. #1 each resource is identified by a persistent identifier We need to properly implement Request Dispatching
- 64. Collections API’s entry point + plural nouns http://api.example.com/v1/contacts
- 65. Collections Flask URL dispatcher allows for variables
@app.route(/<collection>)
def collection(collection):
if collection in DOMAIN.keys():
(...)
abort(404) - 66. Collections Flask URL dispatcher allows for variables
@app.route(/<collection>)
def collection(collection):
if collection in DOMAIN.keys():
(...)
abort(404) - 67. Collections Flask URL dispatcher allows for variables
@app.route(/<collection>)
def collection(collection):
if collection in DOMAIN.keys():
(...)
abort(404) - 68. RegEx by design, collection URLs are plural nouns @app.route(/<regex("[w]*[Ss]"):collection>) def collection(collection): if collection in DOMAIN.keys(): (...) abort(404) regular expressions can be used to better narrow a variable part URL. However...
- 69. RegEx We need to build our own Custom Converter class RegexConverter(BaseConverter): def __init__(self, url_map, *items): super(RegexConverter, self).__init__(url_map) self.regex = items[0] app.url_map.converters[regex] = RegexConverter subclass BaseConverter and pass the new converter to the url_map
- 70. Document Documents are identified by ObjectID http://api.example.com/v1/contacts/4f46445fc88e201858000000 And eventually by an alternative lookup value http://api.example.com/v1/contacts/CUST12345
- 71. Document @app.route(/<regex("[w]*[Ss]"):collection>/<lookup>) @app.route(/<regex("[w]*[Ss]"):collection> /<regex("[a-f0-9]{24}"):object_id>) def document(collection, lookup=None, object_id=None): (...) URL dispatcher handles multiple variables http://api.example.com/v1/contacts/CUST12345
- 72. Document @app.route(/<regex("[w]*[Ss]"):collection>/<lookup>) @app.route(/<regex("[w]*[Ss]"):collection> /<regex("[a-f0-9]{24}"):object_id>) def document(collection, lookup=None, object_id=None): (...) and of course it also handles multiple RegEx variables http://api.example.com/v1/contacts/4f46445fc88e201858000000
- 73. Document @app.route(/<regex("[w]*[Ss]"):collection>/<lookup>) @app.route(/<regex("[w]*[Ss]"):collection> /<regex("[a-f0-9]{24}"):object_id>) def document(collection, lookup=None, object_id=None): (...) Different URLs can be dispatched to the same function just by piling up @app.route decorators.
- 74. #2 representation of resources via media types JSON, XML or any other valid internet media type dep ends on the reque st and not the identifier
- 75. Accepted Media Types mapping supported media types to corresponding renderer functions mime_types = {json_renderer: (application/json,), xml_renderer: (application/xml, text/xml, application/x-xml,)} JSON rendering function
- 76. Accepted Media Types mapping supported media types to corresponding renderer functions mime_types = {json_renderer: (application/json,), xml_renderer: (application/xml, text/xml, application/x-xml,)} corresponding JSON internet media type
- 77. Accepted Media Types mapping supported media types to corresponding renderer functions mime_types = {json_renderer: (application/json,), xml_renderer: (application/xml, text/xml, application/x-xml,)} XML rendering function
- 78. Accepted Media Types mapping supported media types to corresponding renderer functions mime_types = {json_renderer: (application/json,), xml_renderer: (application/xml, text/xml, application/x-xml,)} corresponding XML internet media types
- 79. JSON Render datetimes and ObjectIDs call for further tinkering class APIEncoder(json.JSONEncoder): def default(self, obj): if isinstance(obj, datetime.datetime): return date_to_str(obj) elif isinstance(obj, ObjectId): return str(obj) return json.JSONEncoder.default(self, obj) def json_renderer(**data): return json.dumps(data, cls=APIEncoder) renderer function mapped to the appication/json media type
- 80. JSON Render datetimes and ObjectIDs call for further tinkering class APIEncoder(json.JSONEncoder): def default(self, obj): if isinstance(obj, datetime.datetime): return date_to_str(obj) elif isinstance(obj, ObjectId): return str(obj) return json.JSONEncoder.default(self, obj) def json_renderer(**data): return json.dumps(data, cls=APIEncoder) standard json encoding is not enough, we need a specialized JSONEncoder
- 81. JSON Render datetimes and ObjectIDs call for further tinkering class APIEncoder(json.JSONEncoder): def default(self, obj): if isinstance(obj, datetime.datetime): return date_to_str(obj) elif isinstance(obj, ObjectId): return str(obj) return json.JSONEncoder.default(self, obj) def json_renderer(**data): return json.dumps(data, cls=APIEncoder) Python datetimes are encoded as RFC 1123 strings: “Wed, 06 Jun 2012 14:19:53 UTC”
- 82. JSON Render datetimes and ObjectIDs call for further tinkering class APIEncoder(json.JSONEncoder): def default(self, obj): if isinstance(obj, datetime.datetime): return date_to_str(obj) elif isinstance(obj, ObjectId): return str(obj) return json.JSONEncoder.default(self, obj) def json_renderer(**data): return json.dumps(data, cls=APIEncoder) Mongo ObjectId data types are encoded as strings: “4f46445fc88e201858000000”
- 83. JSON Render datetimes and ObjectIDs call for further tinkering class APIEncoder(json.JSONEncoder): def default(self, obj): if isinstance(obj, datetime.datetime): return date_to_str(obj) elif isinstance(obj, ObjectId): return str(obj) return json.JSONEncoder.default(self, obj) def json_renderer(**data): return json.dumps(data, cls=APIEncoder) we let json/simplejson handle the other data types
- 84. Rendering Render to JSON or XML and get WSGI response object def prep_response(dct, status=200): mime, render = get_best_mime() rendered = globals()[render](**dct) resp = make_response(rendered, status) resp.mimetype = mime return resp best match between request Accept header and media types supported by the service
- 85. Rendering Render to JSON or XML and get WSGI response object def prep_response(dct, status=200): mime, render = get_best_mime() rendered = globals()[render](**dct) resp = make_response(rendered, status) resp.mimetype = mime return resp call the appropriate render function and retrieve the encoded JSON or XML
- 86. Rendering Render to JSON or XML and get WSGI response object def prep_response(dct, status=200): mime, render = get_best_mime() rendered = globals()[render](**dct) resp = make_response(rendered, status) resp.mimetype = mime return resp flask’s make_response() returns a WSGI response object wich we can use to attach headers
- 87. Rendering Render to JSON or XML and get WSGI response object def prep_response(dct, status=200): mime, render = get_best_mime() rendered = globals()[render](**dct) resp = make_response(rendered, status) resp.mimetype = mime return resp and finally, we set the appropriate mime type in the response header
- 88. Flask-MimeRender “Python module for RESTful resource representation using MIME Media-Types and the Flask Microframework” !"!#"$%&((#)(%*+,",-.-$/-.
- 89. Flask-MimeRender Render Functions render_json = jsonify render_xml = lambda message: <message>%s</message> % message render_txt = lambda message: message render_html = lambda message: <html><body>%s</body></html> % message
- 90. Flask-MimeRender then you just decorate your end-point function @app.route(/) @mimerender( default = html, html = render_html, xml = render_xml, json = render_json, txt = render_txt ) def index(): if request.method == GET: return {message: Hello, World!}
- 91. Flask-MimeRender Requests $ curl -H "Accept: application/html" example.com/ <html><body>Hello, World!</body></html> $ curl -H "Accept: application/xml" example.com/ <message>Hello, World!</message> $ curl -H "Accept: application/json" example.com/ {message:Hello, World!} $ curl -H "Accept: text/plain" example.com/ Hello, World!
- 92. #3 resource manipulation through HTTP verbs “GET, POST, PUT, DELETE and all that mess”
- 93. HTTP Methods Verbs are handled along with URL routing @app.route(/<collection>, methods=[GET, POST]) def collection(collection): if collection in DOMAIN.keys(): if request.method == GET: return get_collection(collection) elif request.method == POST: return post(collection) abort(404) accepted HTTP verbs a PUT will throw a 405 Command Not Allowed
- 94. HTTP Methods Verbs are handled along with URL routing @app.route(/<collection>, methods=[GET, POST]) def collection(collection): if collection in DOMAIN.keys(): if request.method == GET: return get_collection(collection) elif request.method == POST: return post(collection) abort(404) the global request object provides access to clients’ request headers
- 95. HTTP Methods Verbs are handled along with URL routing @app.route(/<collection>, methods=[GET, POST]) def collection(collection): if collection in DOMAIN.keys(): if request.method == GET: return get_collection(collection) elif request.method == POST: return post(collection) abort(404) we respond to a GET request for a ‘collection’ resource
- 96. HTTP Methods Verbs are handled along with URL routing @app.route(/<collection>, methods=[GET, POST]) def collection(collection): if collection in DOMAIN.keys(): if request.method == GET: return get_collection(collection) elif request.method == POST: return post(collection) abort(404) and here we respond to a POST request. Handling HTTP methods is easy!
- 97. CRUD via REST Acttion HTTP Verb Context Collection/ Get GET Document Create POST Collection Update PATCH* Document Delete DELETE Document * WTF?
- 98. GET Retrieve Multiple Documents (accepting Queries) http://api.example.com/v1/contacts?where={“age”: {“$gt”: 20}}
- 99. Collection GET http://api.example.com/v1/contacts?where={“age”: {“$gt”: 20}} def get_collection(collection): where = request.args.get(where) if where: args[spec] = json.loads(where, object_hook=datetime_parser) (...) response = {} documents = [] cursor = db(collection).find(**args) for document in cursor: documents.append(document) response[collection] = documents request.args returns the return prep_response(response) original URI’s query definition, in our example: where = {“age”: {“$gt”: 20}}
- 100. Collection GET http://api.example.com/v1/contacts?where={“age”: {“$gt”: 20}} def get_collection(collection): where = request.args.get(where) if where: args[spec] = json.loads(where, object_hook=datetime_parser) (...) response = {} documents = [] cursor = db(collection).find(**args) for document in cursor: documents.append(document) as the query already comes in as a Mongo expression: response[collection] = documents return prep_response(response) {“age”: {“$gt”: 20}} we simply convert it to JSON.
- 101. Collection GET http://api.example.com/v1/contacts?where={“age”: {“$gt”: 20}} def get_collection(collection): where = request.args.get(where) if where: args[spec] = json.loads(where, object_hook=datetime_parser) (...) response = {} documents = [] cursor = db(collection).find(**args) for document in cursor: documents.append(document) response[collection] = documents return prep_response(response) String-to-datetime conversion is obtained via the object_hook mechanis
- 102. Collection GET http://api.example.com/v1/contacts?where={“age”: {“$gt”: 20}} def get_collection(collection): where = request.args.get(where) if where: args[spec] = json.loads(where, object_hook=datetime_parser) (...) response = {} documents = [] cursor = db(collection).find(**args) for document in cursor: documents.append(document) response[collection] = documents find() accepts a python dict return prep_response(response) as query expression, and returns a cursor we can iterate
- 103. Collection GET http://api.example.com/v1/contacts?where={“age”: {“$gt”: 20}} def get_collection(collection): where = request.args.get(where) if where: args[spec] = json.loads(where, object_hook=datetime_parser) (...) response = {} documents = [] cursor = db(collection).find(**args) for document in cursor: documents.append(document) response[collection] = documents return prep_response(response) finally, we encode the response dict with the requested MIME media-type
- 104. Interlude On encoding JSON dates
- 105. On encoding JSON dates • We don’t want to force metadata into JSON representation: (“updated”: “$date: Thu 1, ..”) • Likewise, epochs are not an option • We are aiming for a broad solution not relying on the knoweldge of the current domain
- 106. uy e g ind th eh is b ed R Because, you know
- 107. Parsing JSON dates this is what I came out with >>> source = {"updated": "Thu, 1 Mar 2012 10:00:49 UTC"} >>> dct = json.loads(source, object_hook=datetime_parser) >>> dct {uupdated: datetime.datetime(2012, 3, 1, 10, 0, 49)} def datetime_parser(dct): for k, v in dct.items(): if isinstance(v, basestring) and re.search(" UTC", v): try: dct[k] = datetime.datetime.strptime(v, DATE_FORMAT) except: pass return dct object_hook is usually used to deserialize JSON to classes (rings a ORM bell?)
- 108. Parsing JSON dates this is what I came out with >>> source = {"updated": "Thu, 1 Mar 2012 10:00:49 UTC"} >>> dct = json.loads(source, object_hook=datetime_parser) >>> dct {uupdated: datetime.datetime(2012, 3, 1, 10, 0, 49)} def datetime_parser(dct): for k, v in dct.items(): if isinstance(v, basestring) and re.search(" UTC", v): try: dct[k] = datetime.datetime.strptime(v, DATE_FORMAT) except: pass return dct the resulting dct now has datetime values instead of string representations of dates
- 109. Parsing JSON dates this is what I came out with >>> source = {"updated": "Thu, 1 Mar 2012 10:00:49 UTC"} >>> dct = json.loads(source, object_hook=datetime_parser) >>> dct {uupdated: datetime.datetime(2012, 3, 1, 10, 0, 49)} def datetime_parser(dct): for k, v in dct.items(): if isinstance(v, basestring) and re.search(" UTC", v): try: dct[k] = datetime.datetime.strptime(v, DATE_FORMAT) except: pass return dct the function receives a dict representing the decoded JSON
- 110. Parsing JSON dates this is what I came out with >>> source = {"updated": "Thu, 1 Mar 2012 10:00:49 UTC"} >>> dct = json.loads(source, object_hook=datetime_parser) >>> dct {uupdated: datetime.datetime(2012, 3, 1, 10, 0, 49)} def datetime_parser(dct): for k, v in dct.items(): if isinstance(v, basestring) and re.search(" UTC", v): try: dct[k] = datetime.datetime.strptime(v, DATE_FORMAT) except: pass return dct strings matching the RegEx (which probably should be better defined)...
- 111. Parsing JSON dates this is what I came out with >>> source = {"updated": "Thu, 1 Mar 2012 10:00:49 UTC"} >>> dct = json.loads(source, object_hook=datetime_parser) >>> dct {uupdated: datetime.datetime(2012, 3, 1, 10, 0, 49)} def datetime_parser(dct): for k, v in dct.items(): if isinstance(v, basestring) and re.search(" UTC", v): try: dct[k] = datetime.datetime.strptime(v, DATE_FORMAT) except: pass return dct ...are converted to datetime values
- 112. Parsing JSON dates this is what I came out with >>> source = {"updated": "Thu, 1 Mar 2012 10:00:49 UTC"} >>> dct = json.loads(source, object_hook=datetime_parser) >>> dct {uupdated: datetime.datetime(2012, 3, 1, 10, 0, 49)} def datetime_parser(dct): for k, v in dct.items(): if isinstance(v, basestring) and re.search(" UTC", v): try: dct[k] = datetime.datetime.strptime(v, DATE_FORMAT) except: pass return dct if conversion fails we assume that we are dealing a normal, legit string
- 113. PATCH Editing a Resource
- 114. Why not PUT? • PUT means resource creation or replacement at a given URL • PUT does not allow for partial updates of a resource • 99% of the time we are updating just one or two fields • We don’t want to send complete representations of the document we are updating • Mongo allows for atomic updates and we want to take advantage of that
- 115. ‘atomic’ PUT updates are ok when each field is itself a resource http://api.example.com/v1/contacts/<id>/address
- 116. Enter PATCH “This specification defines the new method, PATCH, which is used to apply partial modifications to a resource.” RFC5789
- 117. PATCH • send a “patch document” with just the changes to be applied to the document • saves bandwidth and reduces traffic • it’s been around since 1995 • it is a RFC Proposed Standard • Widely adopted (will replace PUT in Rails 4.0) • clients not supporting it can fallback to POST with ‘X-HTTP-Method-Override: PATCH’ header tag
- 118. PATCHing def patch_document(collection, original): docs = parse_request(request.form) if len(docs) > 1: abort(400) request.form returns a dict with request form data. key, value = docs.popitem() response_item = {} object_id = original[ID_FIELD] # Validation validate(value, collection, object_id) response_item[validation] = value[validation] if value[validation][response] != VALIDATION_ERROR: # Perform the update updates = {"$set": value[doc]} db(collection).update({"_Id": ObjectId(object_id)}, updates) response_item[ID_FIELD] = object_id return prep_response(response_item)
- 119. PATCHing def patch_document(collection, original): docs = parse_request(request.form) if len(docs) > 1: abort(400) we aren’t going to accept more than one document here key, value = docs.popitem() response_item = {} object_id = original[ID_FIELD] # Validation validate(value, collection, object_id) response_item[validation] = value[validation] if value[validation][response] != VALIDATION_ERROR: # Perform the update updates = {"$set": value[doc]} db(collection).update({"_Id": ObjectId(object_id)}, updates) response_item[ID_FIELD] = object_id return prep_response(response_item)
- 120. PATCHing def patch_document(collection, original): docs = parse_request(request.form) if len(docs) > 1: retrieve the original abort(400) document ID, will be used by key, value = docs.popitem() the update command response_item = {} object_id = original[ID_FIELD] # Validation validate(value, collection, object_id) response_item[validation] = value[validation] if value[validation][response] != VALIDATION_ERROR: # Perform the update updates = {"$set": value[doc]} db(collection).update({"_Id": ObjectId(object_id)}, updates) response_item[ID_FIELD] = object_id return prep_response(response_item)
- 121. PATCHing def patch_document(collection, original): docs = parse_request(request.form) if len(docs) > 1: abort(400) validate the updates key, value = docs.popitem() response_item = {} object_id = original[ID_FIELD] # Validation validate(value, collection, object_id) response_item[validation] = value[validation] if value[validation][response] != VALIDATION_ERROR: # Perform the update updates = {"$set": value[doc]} db(collection).update({"_Id": ObjectId(object_id)}, updates) response_item[ID_FIELD] = object_id return prep_response(response_item)
- 122. PATCHing def patch_document(collection, original): docs = parse_request(request.form) if len(docs) > 1: abort(400) add validation results to the response dictionary key, value = docs.popitem() response_item = {} object_id = original[ID_FIELD] # Validation validate(value, collection, object_id) response_item[validation] = value[validation] if value[validation][response] != VALIDATION_ERROR: # Perform the update updates = {"$set": value[doc]} db(collection).update({"_Id": ObjectId(object_id)}, updates) response_item[ID_FIELD] = object_id return prep_response(response_item)
- 123. PATCHing def patch_document(collection, original): docs = parse_request(request.form) $set accepts a dict if len(docs) > 1: abort(400) with the updates for the db eg: {“active”: False}. key, value = docs.popitem() response_item = {} object_id = original[ID_FIELD] # Validation validate(value, collection, object_id) response_item[validation] = value[validation] if value[validation][response] != VALIDATION_ERROR: # Perform the update updates = {"$set": value[doc]} db(collection).update({"_Id": ObjectId(object_id)}, updates) response_item[ID_FIELD] = object_id return prep_response(response_item)
- 124. PATCHing def patch_document(collection, original): docs = parse_request(request.form) mongo update() method if len(docs) > 1: commits updates to the abort(400) database. Updates are key, value = docs.popitem() atomic. response_item = {} object_id = original[ID_FIELD] # Validation validate(value, collection, object_id) response_item[validation] = value[validation] if value[validation][response] != VALIDATION_ERROR: # Perform the update updates = {"$set": value[doc]} db(collection).update({"_Id": ObjectId(object_id)}, updates) response_item[ID_FIELD] = object_id return prep_response(response_item)
- 125. PATCHing def patch_document(collection, original): docs = parse_request(request.form) if len(docs) > 1: udpate() takes the unique Id abort(400) of the document andthe update expression ($set) key, value = docs.popitem() response_item = {} object_id = original[ID_FIELD] # Validation validate(value, collection, object_id) response_item[validation] = value[validation] if value[validation][response] != VALIDATION_ERROR: # Perform the update updates = {"$set": value[doc]} db(collection).update({"_Id": ObjectId(object_id)}, updates) response_item[ID_FIELD] = object_id return prep_response(response_item)
- 126. PATCHing def patch_document(collection, original): docs = parse_request(request.form) if len(docs) > 1: as always, our response abort(400) dictionary is returned with proper encding key, value = docs.popitem() response_item = {} object_id = original[ID_FIELD] # Validation validate(value, collection, object_id) response_item[validation] = value[validation] if value[validation][response] != VALIDATION_ERROR: # Perform the update updates = {"$set": value[doc]} db(collection).update({"_Id": ObjectId(object_id)}, updates) response_item[ID_FIELD] = object_id return prep_response(response_item)
- 127. POST Creating Resources
- 128. POSTing def post(collection): docs = parse_request(request.form) response = {} for key, item in docs.items(): response_item = {} validate(item, collection) if item[validation][response] != VALIDATION_ERROR: document = item[doc] response_item[ID_FIELD] = db(collection).insert(document) response_item[link] = get_document_link(collection, response_item[ID_FIELD]) response_item[validation] = item[validation] response[key] = response_item return {response: response} we accept multiple documents (remember, we are at collection level here)
- 129. POSTing def post(collection): docs = parse_request(request.form) response = {} for key, item in docs.items(): response_item = {} validate(item, collection) if item[validation][response] != VALIDATION_ERROR: document = item[doc] response_item[ID_FIELD] = db(collection).insert(document) response_item[link] = get_document_link(collection, response_item[ID_FIELD]) response_item[validation] = item[validation] response[key] = response_item return {response: response} we loop through the documents to be inserted
- 130. POSTing def post(collection): docs = parse_request(request.form) response = {} for key, item in docs.items(): response_item = {} validate(item, collection) if item[validation][response] != VALIDATION_ERROR: document = item[doc] response_item[ID_FIELD] = db(collection).insert(document) response_item[link] = get_document_link(collection, response_item[ID_FIELD]) response_item[validation] = item[validation] response[key] = response_item return {response: response} perform validation on the document
- 131. POSTing def post(collection): docs = parse_request(request.form) response = {} for key, item in docs.items(): response_item = {} validate(item, collection) if item[validation][response] != VALIDATION_ERROR: document = item[doc] response_item[ID_FIELD] = db(collection).insert(document) response_item[link] = get_document_link(collection, response_item[ID_FIELD]) response_item[validation] = item[validation] response[key] = response_item return {response: response} push document and get its ObjectId back from Mongo. like other CRUD operations, inserting is trivial in mongo.
- 132. POSTing def post(collection): docs = parse_request(request.form) response = {} for key, item in docs.items(): response_item = {} validate(item, collection) if item[validation][response] != VALIDATION_ERROR: document = item[doc] response_item[ID_FIELD] = db(collection).insert(document) response_item[link] = get_document_link(collection, response_item[ID_FIELD]) response_item[validation] = item[validation] response[key] = response_item return {response: response} a direct link to the resource we just created is added to the response
- 133. POSTing def post(collection): docs = parse_request(request.form) response = {} for key, item in docs.items(): response_item = {} validate(item, collection) if item[validation][response] != VALIDATION_ERROR: document = item[doc] response_item[ID_FIELD] = db(collection).insert(document) response_item[link] = get_document_link(collection, response_item[ID_FIELD]) response_item[validation] = item[validation] response[key] = response_item return {response: response} validation result is always returned to the client, even if the doc has not been inserted
- 134. POSTing def post(collection): docs = parse_request(request.form) response = {} for key, item in docs.items(): response_item = {} validate(item, collection) if item[validation][response] != VALIDATION_ERROR: document = item[doc] response_item[ID_FIELD] = db(collection).insert(document) response_item[link] = get_document_link(collection, response_item[ID_FIELD]) response_item[validation] = item[validation] response[key] = response_item return {response: response} standard response enconding applied
- 135. Data Validation We still need to validate incoming data
- 136. Data Validation DOMAIN = {} DOMAIN[contacts] = { secondary_id: name, fields: { name: { data_type: string, required: True, unique: True, max_length: 120, min_length: 1 }, DOMAIN is a Python dict containing our validation rules and schema structure
- 137. Data Validation DOMAIN = {} DOMAIN[contacts] = { secondary_id: name, fields: { name: { data_type: string, required: True, unique: True, max_length: 120, min_length: 1 }, every resource (collection) maintained by the API has a key in DOMAIN
- 138. Data Validation DOMAIN = {} DOMAIN[contacts] = { secondary_id: name, fields: { name: { data_type: string, required: True, unique: True, max_length: 120, min_length: 1 }, if the resource allows for a secondary lookup field, we define it here
- 139. Data Validation DOMAIN = {} DOMAIN[contacts] = { secondary_id: name, fields: { name: { data_type: string, required: True, unique: True, max_length: 120, min_length: 1 }, known fields go in the fields dict
- 140. Data Validation DOMAIN = {} DOMAIN[contacts] = { secondary_id: name, fields: { name: { data_type: string, required: True, unique: True, max_length: 120, min_length: 1 }, validation rules for ‘name’ field. data_type is mostly needed to process datetimes and currency values
- 141. Data Validation (...) iban: { data_type: string, custom_validation: { module: customvalidation, function: validate_iban } } (...) we can define custom validation functions when the need arises
- 142. Data Validation (...) contact_type: { data_type: array, allowed_values: [ client, agent, supplier, area manager, vector ] } (...) or we can define our own custom data types...
- 143. Data Validation (...) contact_type: { data_type: array, allowed_values: [ client, agent, supplier, area manager, vector ] } (...) ... like the array, which allows us to define a list of accepted values for the field
- 144. I will spare you the validation function It’s pretty simple really
- 145. Hey but! You’re building your own ORM! Just a thin validation layer on which I have total control AKA So What?
- 146. #4 Caching and concurrency control resource representation describes how when and if it can be used, discarded or re-fetched
- 147. Driving conditional requests Servers use Last-Modified and ETag response headers to drive conditional requests
- 148. Last-Modified Generally considered a weak validator since it has a one-second resolution “Wed, 06 Jun 2012 14:19:53 UTC”
- 149. ETag Entity Tag is a strong validator since its value can be changed every time the server modifies the representation 7a9f477cde424cf93a7db20b69e05f7b680b7f08
- 150. On ETags • Clients should be able to use ETag to compare representations of a resouce • An ETag is supposed to be like an object’s hash code. • Actually, some web frameworks and a lot of implementations do just that • ETag computed on an entire representation of the resource may become a performance bottleneck
- 151. Last-Modified or ETag? You can use either or both. Consider the types of client consuming your service. Hint: use both.
- 152. Validating cached representations Clients use If-Modified-Since and If-None-Match in request headers for validating cached representations
- 153. If-Mod-Since & ETag def get_document(collection, object_id=None, lookup=None): response = {} document = find_document(collection, object_id, lookup) if document: etag = get_etag(document) header_etag = request.headers.get(If-None-Match) if header_etag and header_etag == etag: return prep_response(dict(), status=304) if_modified_since = request.headers.get(If-Modified-Since) if if_modified_since: last_modified = document[LAST_UPDATED] if last_modified <= if_modified_since: return prep_response(dict(), status=304) response[collection.rstrip(s)] = document return prep_response(response, last_modified, etag) abort(404) retrieve the document from the database
- 154. If-Mod-Since & ETag def get_document(collection, object_id=None, lookup=None): response = {} document = find_document(collection, object_id, lookup) if document: etag = get_etag(document) header_etag = request.headers.get(If-None-Match) if header_etag and header_etag == etag: return prep_response(dict(), status=304) if_modified_since = request.headers.get(If-Modified-Since) if if_modified_since: last_modified = document[LAST_UPDATED] if last_modified <= if_modified_since: return prep_response(dict(), status=304) response[collection.rstrip(s)] compute ETag for the current = document return prep_response(response, last_modified, etag) representation. We test ETag abort(404) first, as it is a stronger validator
- 155. If-Mod-Since & ETag def get_document(collection, object_id=None, lookup=None): response = {} document = find_document(collection, object_id, lookup) if document: etag = get_etag(document) header_etag = request.headers.get(If-None-Match) if header_etag and header_etag == etag: return prep_response(dict(), status=304) if_modified_since = request.headers.get(If-Modified-Since) if if_modified_since: last_modified = document[LAST_UPDATED] if last_modified <= if_modified_since: return prep_response(dict(), status=304) response[collection.rstrip(s)] = document return prep_response(response, last_modified, etag) abort(404) retrieve If-None-Match ETag from request header
- 156. If-Mod-Since & ETag def get_document(collection, object_id=None, lookup=None): response = {} document = find_document(collection, object_id, lookup) if document: etag = get_etag(document) header_etag = request.headers.get(If-None-Match) if header_etag and header_etag == etag: return prep_response(dict(), status=304) if_modified_since = request.headers.get(If-Modified-Since) if if_modified_since: last_modified = document[LAST_UPDATED] if last_modified <= if_modified_since: return prep_response(dict(), status=304) response[collection.rstrip(s)] = document return prep_response(response, last_modified, etag) abort(404) if client and server representations match, return a 304 Not Modified
- 157. If-Mod-Since & ETag def get_document(collection, object_id=None, lookup=None): response = {} document = find_document(collection, object_id, lookup) if document: etag = get_etag(document) header_etag = request.headers.get(If-None-Match) if header_etag and header_etag == etag: return prep_response(dict(), status=304) if_modified_since = request.headers.get(If-Modified-Since) if if_modified_since: last_modified = document[LAST_UPDATED] if last_modified <= if_modified_since: return prep_response(dict(), status=304) response[collection.rstrip(s)] = document return prep_response(response, last_modified, if the resource likewise, etag) abort(404) has not been modified since If-Modifed-Since, return 304 Not Modified
- 158. Concurrency control Clients use If-Unmodified-Since and If-Match in request headers as preconditions for concurrency control
- 159. Concurrency control Create/Update/Delete are controlled by ETag def edit_document(collection, object_id, method): document = find_document(collection, object_id) if document: header_etag = request.headers.get(If-Match) if header_etag is None: return prep_response(If-Match missing from request header, status=403) if header_etag != get_etag(document[LAST_UPDATED]): # Precondition failed abort(412) else: if method in (PATCH, POST): return patch_document(collection, document) elif method == DELETE: return delete_document(collection, object_id) else: abort(404) retrieve client’s If-Match ETag from the request header
- 160. Concurrency control Create/Update/Delete are controlled by ETag def edit_document(collection, object_id, method): document = find_document(collection, object_id) if document: header_etag = request.headers.get(If-Match) if header_etag is None: return prep_response(If-Match missing from request header, status=403) if header_etag != get_etag(document[LAST_UPDATED]): # Precondition failed abort(412) else: if method in (PATCH, POST): return patch_document(collection, document) elif method == DELETE: return delete_document(collection, object_id) else: abort(404) editing is forbidden if ETag is not provided
- 161. Concurrency control Create/Update/Delete are controlled by ETag def edit_document(collection, object_id, method): document = find_document(collection, object_id) if document: header_etag = request.headers.get(If-Match) if header_etag is None: return prep_response(If-Match missing from request header, status=403) if header_etag != get_etag(document[LAST_UPDATED]): # Precondition failed abort(412) else: if method in (PATCH, POST): return patch_document(collection, document) elif method == DELETE: return delete_document(collection, object_id) else: client and server abort(404) representations don’t match. Precondition failed.
- 162. Concurrency control Create/Update/Delete are controlled by ETag def edit_document(collection, object_id, method): document = find_document(collection, object_id) if document: header_etag = request.headers.get(If-Match) if header_etag is None: client and server representation match, return prep_response(If-Match missing from request header, status=403) go ahead with the edit if header_etag != get_etag(document[LAST_UPDATED]): # Precondition failed abort(412) else: if method in (PATCH, POST): return patch_document(collection, document) elif method == DELETE: return delete_document(collection, object_id) else: abort(404)
- 163. Sending cache & concurrency directives back to clients
- 164. Cache & Concurrency def prep_response(dct, last_modified=None, etag=None, status=200): (...) resp.headers.add(Cache-Control, max-age=%s,must-revalidate & 30) resp.expires = time.time() + 30 if etag: resp.headers.add(ETag, etag) if last_modified: resp.headers.add(Last-Modified, date_to_str(last_modified)) return resp encodes ‘dct’ according to client’s accepted MIME Data-Type (click here see that slide)
- 165. Cache & Concurrency def prep_response(dct, last_modified=None, etag=None, status=200): (...) resp.headers.add(Cache-Control, max-age=%s,must-revalidate & 30) resp.expires = time.time() + 30 if etag: resp.headers.add(ETag, etag) if last_modified: resp.headers.add(Last-Modified, date_to_str(last_modified)) return resp Cache-Control, a directive for HTTP/1.1 clients (and later) -RFC2616
- 166. Cache & Concurrency def prep_response(dct, last_modified=None, etag=None, status=200): (...) resp.headers.add(Cache-Control, max-age=%s,must-revalidate & 30) resp.expires = time.time() + 30 if etag: resp.headers.add(ETag, etag) if last_modified: resp.headers.add(Last-Modified, date_to_str(last_modified)) return resp Expires, a directive for HTTP/1.0 clients
- 167. Cache & Concurrency def prep_response(dct, last_modified=None, etag=None, status=200): (...) resp.headers.add(Cache-Control, max-age=%s,must-revalidate & 30) resp.expires = time.time() + 30 if etag: resp.headers.add(ETag, etag) if last_modified: resp.headers.add(Last-Modified, date_to_str(last_modified)) return resp ETag. Notice that we don’t compute it on the rendered representation, this is by design.
- 168. Cache & Concurrency def prep_response(dct, last_modified=None, etag=None, status=200): (...) resp.headers.add(Cache-Control, max-age=%s,must-revalidate & 30) resp.expires = time.time() + 30 if etag: resp.headers.add(ETag, etag) if last_modified: resp.headers.add(Last-Modified, date_to_str(last_modified)) return resp And finally, we add the Last-Modified header tag.
- 169. Cache & Concurrency def prep_response(dct, last_modified=None, etag=None, status=200): (...) resp.headers.add(Cache-Control, max-age=%s,must-revalidate & 30) resp.expires = time.time() + 30 if etag: resp.headers.add(ETag, etag) if last_modified: resp.headers.add(Last-Modified, date_to_str(last_modified)) return resp the response object is now complete and ready to be returned to the client
- 170. that ’s o long ne acro ass nym #5 HATEOAS “Hypertext As The Engine Of Application State”
- 171. HATEOAS in a Nutshell • clients interact entirely through hypermedia provided dynamically by the server • clients need no prior knowledge about how to interact with the server • clients access an application through a single well known URL (the entry point) • All future actions the clients may take are discovered within resource representations returned from the server
- 172. It’s all about Links resource representation includes links to related resources
- 173. Collection { Representation "links":[ "<link rel=parent title=home href=http://api.example.com/ />", "<link rel=collection title=contacts href=http://api.example.com/Contacts />", "<link rel=next title=next page href=http://api.example.com/Contacts?page=2 />" ], "contacts":[ { "updated":"Wed, 06 Jun 2012 14:19:53 UTC", "name":"Jon Doe", "age": 27, "etag":"7a9f477cde424cf93a7db20b69e05f7b680b7f08", "link":"<link rel=self title=Contact every resource href=http://api.example.com/Contacts/ 4f46445fc88e201858000000 />", representation provides a "_id":"4f46445fc88e201858000000", links section with }, navigational info for ] clients }
- 174. Collection { Representation "links":[ "<link rel=parent title=home href=http://api.example.com/ />", "<link rel=collection title=contacts href=http://api.example.com/Contacts />", "<link rel=next title=next page href=http://api.example.com/Contacts?page=2 />" ], "contacts":[ { "updated":"Wed, 06 Jun 2012 14:19:53 UTC", "name":"Jon Doe", "age": 27, "etag":"7a9f477cde424cf93a7db20b69e05f7b680b7f08", "link":"<link rel=self title=Contact href=http://api.example.com/Contacts/ attribute provides the rel 4f46445fc88e201858000000 />", the relationship between the "_id":"4f46445fc88e201858000000", }, linked resource and the one ] currently represented }
- 175. Collection { Representation "links":[ "<link rel=parent title=home href=http://api.example.com/ />", "<link rel=collection title=contacts href=http://api.example.com/Contacts />", "<link rel=next title=next page href=http://api.example.com/Contacts?page=2 />" ], "contacts":[ { "updated":"Wed, 06 Jun 2012 14:19:53 UTC", "name":"Jon Doe", "age": 27, "etag":"7a9f477cde424cf93a7db20b69e05f7b680b7f08", "link":"<link rel=self title=Contact the title attribute provides href=http://api.example.com/Contacts/ 4f46445fc88e201858000000 />", a tag (or description) for "_id":"4f46445fc88e201858000000", the linked resource. Could }, be used as a caption for a ] client button. }
- 176. Collection { Representation "links":[ "<link rel=parent title=home href=http://api.example.com/ />", "<link rel=collection title=contacts href=http://api.example.com/Contacts />", "<link rel=next title=next page href=http://api.example.com/Contacts?page=2 />" ], "contacts":[ { "updated":"Wed, 06 Jun 2012 14:19:53 UTC", "name":"Jon Doe", "age": 27, "etag":"7a9f477cde424cf93a7db20b69e05f7b680b7f08", "link":"<link rel=self title=Contact href=http://api.example.com/Contacts/ attribute provides the href 4f46445fc88e201858000000 />", and absolute path to the "_id":"4f46445fc88e201858000000", }, resource (the “permanent ] identifier” per REST def.) }
- 177. Collection { Representation "links":[ "<link rel=parent title=home href=http://api.example.com/ />", "<link rel=collection title=contacts every resource listed href=http://api.example.com/Contacts />", exposes its own link, which "<link rel=next title=next page will allow the client to href=http://api.example.com/Contacts?page=2 />" ], perform PATCH, DELETE etc. "contacts":[ on the resource { "updated":"Wed, 06 Jun 2012 14:19:53 UTC", "name":"Jon Doe", "age": 27, "etag":"7a9f477cde424cf93a7db20b69e05f7b680b7f08", "link":"<link rel=self title=Contact href=http://api.example.com/Contacts/ 4f46445fc88e201858000000 />", "_id":"4f46445fc88e201858000000", }, ] }
- 178. Collection { Representation "links":[ while we are here, "<link rel=parent title=home href=http://api.example.com/ />", "<link rel=collection title=contacts notice how every resource href=http://api.example.com/Contacts />", its own etag, also exposes "<link rel=next title=next page last-modified date. href=http://api.example.com/Contacts?page=2 />" ], "contacts":[ { "updated":"Wed, 06 Jun 2012 14:19:53 UTC", "name":"Jon Doe", "age": 27, "etag":"7a9f477cde424cf93a7db20b69e05f7b680b7f08", "link":"<link rel=self title=Contact href=http://api.example.com/Contacts/ 4f46445fc88e201858000000 />", "_id":"4f46445fc88e201858000000", }, ] }
- 179. HATEOAS The API entry point (the homepage) @app.route(/, methods=[GET]) def home(): response = {} links = [] for collection in DOMAIN.keys(): links.append("<link rel=child title=%(name)s" "href=%(collectionURI)s />" % {name: collection, collectionURI: collection_URI(collection)}) response[links] = links return response the API homepage responds to GET requests and provides links to its top level resources to the clients
- 180. HATEOAS The API entry point (the homepage) @app.route(/, methods=[GET]) def home(): response = {} links = [] for collection in DOMAIN.keys(): links.append("<link rel=child title=%(name)s" "href=%(collectionURI)s />" % {name: collection, collectionURI: collection_URI(collection)}) response[links] = links return response for every collection of resources...
- 181. HATEOAS The API entry point (the homepage) @app.route(/, methods=[GET]) def home(): response = {} links = [] for collection in DOMAIN.keys(): links.append("<link rel=child title=%(name)s" "href=%(collectionURI)s />" % {name: collection, collectionURI: collection_URI(collection)}) response[links] = links return response ... provide relation, title and link, or the persistent identifier
- 182. Wanna see it running? Hopefully it won’t explode right into my face
- 183. Only complaint I have with Flask so far... Most recent HTTP methods not supported
- 184. 508 NOT MY FAULT Not supported
- 185. 208 WORKS FOR ME Not supported
- 186. en Just kidding! ev ! ’t ke sn jo ti y i
- 187. Open Source it? as a Flask extension maybe?
- 188. Web Resources • Richardson Maturity Model: steps toward the glory of REST by Richard Flowers • RESTful Service Best Practices by Todd Fredrich • What Exactly(lotsRESTful Programming? StackOverflow is of resources) • API Anti-Patterns: How to Avoid Common REST Mistakes by Tomas Vitvar
- 189. Excellent Books
- 190. Excellent Books I’m getting a cut. ish! Iw
- 191. Thank you. @nicolaiarocci