database

Implement Apache CouchDB with Python on Ubuntu 20.04

Introduction

Apache CouchDB is a NoSQL document-based database server that stores data in JSON format. This guide explains how to access CouchDB in Python.

By default, CouchDB ships with an HTTP API for performing database operations. However, you may want to combine the power of the CouchDB database with the rich Python functions to process the data further. This approach is applicable when designing a backend Application Programming Interface (API) without directly exposing end-users to the CouchDB server.

This guide takes you through implementing the Apache CouchDB server with Python on Ubuntu 20.04 server.

Prerequisites

Before you begin:

  1. Deploy an Ubuntu 20.04 server.
  2. Create a non-root sudo user.
  3. Install Apache CouchDb database.

1. Set Up a CouchDb Database

The first step in creating this sample application is initializing the CouchDB database. SSH to your server and use the Linux curl command to execute the following database operations. Remember to replace EXAMPLE_PASSWORD with the correct CouchDB admin password:

  1. Create a sample my_company database.$ curl -X PUT http://admin:EXAMPLE_PASSWORD@127.0.0.1:5984/my_company Output.{"ok":true}
  2. Insert some documents into the my_company database to ensure the database is ready to accept new documents. The following documents contain data for products. That is product_ids, product_names, and the retail_prices.$ curl -H 'Content-Type: application/json' -X PUT http://admin:EXAMPLE_PASSWORD@127.0.0.1:5984/my_company/"1" -d '{"product_name": "RIPPED JEANS" , "retail_price" : 32.50}' $ curl -H 'Content-Type: application/json' -X PUT http://admin:EXAMPLE_PASSWORD@127.0.0.1:5984/my_company/"2" -d '{"product_name": "HIGHT WAIST JEANS" , "retail_price" : 17.25}' $ curl -H 'Content-Type: application/json' -X PUT http://admin:EXAMPLE_PASSWORD@127.0.0.1:5984/my_company/"3" -d '{"product_name": "STRAIGHT CUT JEANS" , "retail_price" : 49.80}' $ curl -H 'Content-Type: application/json' -X PUT http://admin:EXAMPLE_PASSWORD@127.0.0.1:5984/my_company/"4" -d '{"product_name": "COTTON BLEND JEANS" , "retail_price" : 42.35}' $ curl -H 'Content-Type: application/json' -X PUT http://admin:EXAMPLE_PASSWORD@127.0.0.1:5984/my_company/"5" -d '{"product_name": "HIGH-WAIST JEANS" , "retail_price" : 34.80}' Output.{"ok":true,"id":"1","rev":"1-f0458dc03140b0cf8de6d2d2fa46ad72"} {"ok":true,"id":"2","rev":"1-ccf054656db9d18a0fa1361547d12297"} {"ok":true,"id":"3","rev":"1-19b450bee38c284f57b71c12be9a18f4"} {"ok":true,"id":"4","rev":"1-97d8cc255704ebc8de92e7a9a99577dc"} {"ok":true,"id":"5","rev":"1-346b8fbf285fa36473cc081cc377159d"}
  3. Query the my_company database to ensure the documents are in place.$ curl -X GET http://admin:EXAMPLE_PASSWORD@127.0.0.1:5984/my_company/_all_docs?include_docs=true Output.{ "total_rows":5, "offset":0, "rows":[ { "id":"1", "key":"1", "value":{ "rev":"1-f0458dc03140b0cf8de6d2d2fa46ad72" }, "doc":{ "_id":"1", "_rev":"1-f0458dc03140b0cf8de6d2d2fa46ad72", "product_name":"RIPPED JEANS", "retail_price":32.5 } }, { "id":"2", "key":"2", "value":{ "rev":"1-ccf054656db9d18a0fa1361547d12297" }, "doc":{ "_id":"2", "_rev":"1-ccf054656db9d18a0fa1361547d12297", "product_name":"HIGHT WAIST JEANS", "retail_price":17.25 } }, { "id":"3", "key":"3", "value":{ "rev":"1-19b450bee38c284f57b71c12be9a18f4" }, "doc":{ "_id":"3", "_rev":"1-19b450bee38c284f57b71c12be9a18f4", "product_name":"STRAIGHT CUT JEANS", "retail_price":49.8 } }, { "id":"4", "key":"4", "value":{ "rev":"1-97d8cc255704ebc8de92e7a9a99577dc" }, "doc":{ "_id":"4", "_rev":"1-97d8cc255704ebc8de92e7a9a99577dc", "product_name":"COTTON BLEND JEANS", "retail_price":42.35 } }, { "id":"5", "key":"5", "value":{ "rev":"1-346b8fbf285fa36473cc081cc377159d" }, "doc":{ "_id":"5", "_rev":"1-346b8fbf285fa36473cc081cc377159d", "product_name":"HIGH-WAIST JEANS", "retail_price":34.8 } } ] }
  4. Proceed to the next step to set up a project directory and a gateway class for accessing your CouchDB server.

2. Create a Database Class

You should structure your Python source code in modules to enhance code readability and support. Always create a separate directory for the source code to avoid mixing up the application’s and system’s files. For this project, you require a database class module that connects to the CouchDB server. Follow the steps below to create the class:

  1. Create a project directory.$ mkdir project
  2. Switch to the new project directory.$ cd project
  3. Open a new database_gateway.py file in a text editor.$ nano database_gateway.py
  4. Enter the following information into the database_gateway.py file. Replace the values of the db_username, db_password db_host, and db_port, db_name variables with the correct CouchDB database credentials.import couchdb class DatabaseGateway: def db_connect(self): try: db_username = 'admin' db_password = 'EXAMPLE_PASSWORD' db_host = '127.0.0.1' db_port = '5984' db_name = 'my_company' con_string = 'http://' + db_username + ':' + db_password + '@' + db_host + ':' + db_port + '/' server = couchdb.Server(con_string) db_con = server[db_name] return db_con except Exception as e: print(e) return [];
  5. Save and close the database_gateway.py file.

The database_gateway.py file explained:

  1. The import couchdb declaration imports the couchdb module for Python.
  2. The database_gateway.py file contains a single DatabaseGateway class with a single db_connect()` method.class DatabaseGateway: def db_connect(self): ...
  3. Under the db_connect() method, you’re using the CouchDB server’s credentials to create a database connection using the couchdb.Server(con_string) statement. The db_connect() function then returns a reusable database connection using the return db_con statement.
  4. The database_gateway module is ready for use. You can initialize the module in other source code files as follows:import database_gateway dg = database_gateway.DatabaseGateway() db_connect = dg.db_connect()
  5. Proceed to the next step to create another module that allows you to access and manipulate products’ information in the CouchDB server.

3. Create a Resource Class

A production-ready application can have dozens or hundreds of resources. This guide shows you how to create a single products resource for demonstration purposes. This class should allow you to perform the following tasks:

  • Create new products.
  • Read products’ details.
  • Update products’ details.
  • Delete products.

Follow the steps below to create the products class:

  1. Open a new products.py file in a text editor.$ nano products.py
  2. Enter the following information into the products.py file.class Products: def __init__(self, db_con): self.db_con = db_con def create(self, data): try: doc = { '_id': data["_id"], 'product_name': data["product_name"], 'retail_price': data["retail_price"], } resource_id, doc_rev = self.db_con.save(doc) return self.read(resource_id) except Exception as e: print(e) return []; def read(self, resource_id): try: res = [] if resource_id != "": doc = self.db_con[resource_id] res.append(doc) else: for row in self.db_con.view('_all_docs', include_docs=True): doc = row.doc res.append(doc) return res except Exception as e: print(e) return []; def update(self, resource_id, data): try: doc = self.read(resource_id)[0] doc["product_name"] = data["product_name"] doc["retail_price"] = data["retail_price"] self.db_con.save(doc) return self.read(resource_id) except Exception as e: print(e) return []; def delete(self, resource_id): try: self.db_con.delete(self.db_con[resource_id]) return '{Success}' except Exception as e: print(e) return [];
  3. Save and close the products.py file

The products.py file explained:

  1. The products.py file contains a single Products class and a constructor (__init__) method. The constructor method accepts a CouchDb connection (db_con) object when initializing.class Products: def __init__(self, db_con): self.db_con = db_con ...
  2. The Products class has four other methods that focus on database operations. ... def create(self, data): ... def read(self, resource_id): ... def update(self, resource_id, data): ... def delete(self, resource_id): ...
  3. The four methods perform the following functions:
    • create(self, data): This method accepts a data argument containing a new document’s data in JSON format. The create() function then constructs a document using the doc = {} format and passes it to the CouchDB database server using the self.db_con.save(doc) function. The self.db_con is the database connection object passed via the constructor method(__init__(self, db_con)). In the end, the create() function runs a read() function and returns the data from the CouchDB database using the return self.read(resource_id) statement.
    • read(self, resource_id): The read() function takes a resource_id argument containing a unique _id of the document you want to retrieve. Under the read() function the logical if ... else ... statement checks if the resource_id argument contains a value (if resource_id != ""). In case the resource_id is not empty, the read() function retrieves a document from the CouchDB database that matches the resource_id value (doc = self.db_con[resource_id]). Otherwise, the read() function queries the CouchDB database to get all documents using the for row ... loop and the doc = row.doc statements. The read() function finally returns an object containing the documents using the return res statement.
    • update(self, resource_id, data): This function accepts the resource_id of the document you want to update together with the new document’s data. Then, the update() function executes the self.db_con.save(doc) function to update the document. In the end, the update function returns the updated document’s data using the return self.read(resource_id) statement.
    • delete(self, resource_id): This function takes the unique resource_id of the product that you want to delete and runs the CouchDB’s self.db_con.delete(self.db_con[resource_id]) function to remove the document from the database. The delete() function then returns a {Success} after executing the function successfully.
  4. The try...except block allows you to catch and display (print(e)) any errors that may occur during the database operations.... try: ... except Exception as e: print(e) return []; ...
  5. The products resource module is ready. You may invoke it in other project files using the following declaration.import products resource = products.Products(db_connect) resource.create(...) resource.read(...) resource.update(...) resource.delete(...)
  6. Proceed to the next step to code the application’s entry point.

4. Create Application’s Main File

Every Python application should contain the main file that executes when you start the application. This step shows you how to set up a main.py file that uses your custom database_gateway and products modules.

  1. Open a new main.py file in a text editor.$ nano main.py
  2. Enter the following information into the main.py file.import http.server from http import HTTPStatus import socketserver import json import database_gateway import products class httpHandler(http.server.SimpleHTTPRequestHandler): def send_headers(self): self.send_response(HTTPStatus.OK) self.send_header('Content-type', 'application/json') self.end_headers() def get_json_body(self): content_length = int(self.headers['Content-Length']) post_data = self.rfile.read(content_length) json_body = json.loads(post_data) return json_body def get_resource_id(self): resource_id = "" if len(self.path.split("/")) >= 3: resource_id = self.path.split("/")[2] return resource_id def do_POST(self): self.send_headers() dg = database_gateway.DatabaseGateway() db_connect = dg.db_connect() resource = products.Products(db_connect) resp = resource.create(self.get_json_body()) self.wfile.write(bytes(json.dumps(resp, indent = 4) + "\n", "utf8")) def do_GET(self): self.send_headers() dg = database_gateway.DatabaseGateway() db_connect = dg.db_connect() resource = products.Products(db_connect) resp = resource.read(self.get_resource_id()) self.wfile.write(bytes(json.dumps(resp, indent = 4) + "\n", "utf8")) def do_PUT(self): self.send_headers() dg = database_gateway.DatabaseGateway() db_connect = dg.db_connect() resource = products.Products(db_connect) resp = resource.update(self.get_resource_id(), self.get_json_body()) self.wfile.write(bytes(json.dumps(resp, indent = 4) + "\n", "utf8")) def do_DELETE(self): self.send_headers() dg = database_gateway.DatabaseGateway() db_connect = dg.db_connect() resource = products.Products(db_connect) resp = resource.delete(self.get_resource_id()) self.wfile.write(bytes(json.dumps(resp, indent = 4) + "\n", "utf8")) httpd = socketserver.TCPServer(('', 8080), httpHandler) print("HTTP server started...") try: httpd.serve_forever() except KeyboardInterrupt: httpd.server_close() print("The server is stopped.")
  3. Save and close the main.py file.

The main.py file explained:

  1. The import section allows you to declare the Python’s http.server, json, and custom application’s modules (database_gateway and products).import http.server from http import HTTPStatus import socketserver import json import database_gateway import products ...
  2. The httpHandler class defines a handler for the Python’s web server.... class httpHandler(http.server.SimpleHTTPRequestHandler): ...
  3. The following code block starts an HTTP web server that listens for incoming connections on port 8080.... httpd = socketserver.TCPServer(('', 8080), httpHandler) print("HTTP server started...") try: httpd.serve_forever() except KeyboardInterrupt: httpd.server_close() print("The server is stopped.")
  4. The httpHandler class has seven methods.
    • send_headers(self): The method sets the correct headers for the application, including the application/json content type.
    • get_json_body(self): This method reads the HTTP request body using the self.rfile.read(content_length) function and then returns the data in JSON format (json.loads(post_data)). The HTTP POST and PUT methods require the JSON body.
    • get_resource_id(self): This method uses the self.path.split("/") function to split the request URL and retrieve the resource_id for GET and DELETE HTTP methods. For instance, if the application receives a request for the http://localhost:8080/products/3 URL, the function returns 3 as the resource_id.
    • do_POST(self): This method initializes your custom modules (database_gateway and products) to save a document in the CouchDB server using the resource.create(self.get_json_body()) function.
    • do_GET(self): This method uses the custom modules to retrieve data from the CouchDB server using the resource.read(self.get_resource_id()) statement.
    • do_PUT(self): This method passes a JSON body and a resource_id to the products module to update a document’s details using the resource.update(self.get_resource_id(), self.get_json_body()) statement.
    • do_DELETE(self): This method removes a document from a CouchDB server using the resource.delete(self.get_resource_id()) statement.
  5. The self.wfile.write(bytes(json.dumps(resp, indent = 4) + "\n", "utf8")) statement writes a JSON response to the HTTP clients.
  6. You’ve defined the application’s entry point. Follow the next step to test the application’s functions

5. Test the Application

The required source code files for running the application are now ready. This step focuses on downloading the required dependency packages and testing the application.

  1. Begin by updating the package information index.$ sudo apt update
  2. Download the Python pip package, a module for installing Python libraries.$ sudo apt install -y python3-pip
  3. Ensure the Python dependencies are in place.$ sudo apt install --reinstall base-files software-properties-common python3-apt
  4. Use the pip package to install the couchdb package for Python, a library that allows you to talk to the CouchDB database server from a Python code.$ sudo pip install couchdb Output.... Installing collected packages: couchdb Successfully installed couchdb-1.2
  5. Run the application.$ python3 main.py The application starts a web server on port 8080. Do not enter any other command on your active terminal window because the above command has a blocking function.HTTP server started...
  6. Establish a second terminal window and run the following Linux curl commands:
    • Retrieve all documents:$ curl -X GET http://localhost:8080/products Output.[ { "_id": "1", "_rev": "1-f0458dc03140b0cf8de6d2d2fa46ad72", "product_name": "RIPPED JEANS", "retail_price": 32.5 }, { "_id": "2", "_rev": "1-ccf054656db9d18a0fa1361547d12297", "product_name": "HIGHT WAIST JEANS", "retail_price": 17.25 }, { "_id": "3", "_rev": "1-19b450bee38c284f57b71c12be9a18f4", "product_name": "STRAIGHT CUT JEANS", "retail_price": 49.8 }, { "_id": "4", "_rev": "1-97d8cc255704ebc8de92e7a9a99577dc", "product_name": "COTTON BLEND JEANS", "retail_price": 42.35 }, { "_id": "5", "_rev": "1-346b8fbf285fa36473cc081cc377159d", "product_name": "HIGH-WAIST JEANS", "retail_price": 34.8 } ]
    • Retrieve a single document:$ curl -X GET http://localhost:8080/products/3 Output.[ { "_id": "3", "_rev": "1-19b450bee38c284f57b71c12be9a18f4", "product_name": "STRAIGHT CUT JEANS", "retail_price": 49.8 } ]
    • Create a new document:$ curl -X POST http://localhost:8080/products -H 'Content-Type: application/json' -d '{"_id": "6", "product_name": "DELL COMPUTER", "retail_price": "750.50"}' Output.[ { "_id": "6", "_rev": "1-b87e76b3a4feb47304a010ec688d6142", "product_name": "DELL COMPUTER", "retail_price": "750.50" } ]
    • Update a document:$ curl -X PUT http://localhost:8080/products/6 -H 'Content-Type: application/json' -d '{"product_name": "DELL LAPTOP COMPUTER ", "retail_price": "800.45"}' Output.[ { "_id": "6", "_rev": "2-6427cfba12d76461e343e6edd4127c68", "product_name": "DELL LAPTOP COMPUTER ", "retail_price": "800.45" } ]
    • Delete a document:$ curl -X DELETE http://localhost:8080/products/2 Output."{Success}" Try retrieving the document you’ve deleted.$ curl -X GET http://localhost:8080/products/2 Output.[]

Conclusion

This guide implements the Apache CouchDB database server with Python on Ubuntu 20.04 server. This guide shows you how to set up a sample CouchDB database and create some Python modules to perform basic database operations. In the end, you’ve: inserted, updated, read, and updated CouchDB documents using Python and the Linux curl command.

Follow the links below to read more Python tutorials:

Introduction Apache CouchDB is a NoSQL document-based database server that stores data in JSON format. This guide explains how to access CouchDB in Python. By default, CouchDB ships with an HTTP API for performing database operations. However, you may want to combine the power of the CouchDB database with the…

Introduction Apache CouchDB is a NoSQL document-based database server that stores data in JSON format. This guide explains how to access CouchDB in Python. By default, CouchDB ships with an HTTP API for performing database operations. However, you may want to combine the power of the CouchDB database with the…

Leave a Reply

Your email address will not be published. Required fields are marked *