Building a Real Time Chat Application with Neurelo and MongoDB using Python
Author: Parth Parikh (parth@neurelo.com)
Last updated
Author: Parth Parikh (parth@neurelo.com)
Last updated
Chat applications have certainly come a long way. One of the earliest chat applications, EMISARI, was commissioned by the US government in 1971 to facilitate message exchanges among officials during emergencies. At that time, it operated from a teletypewriter terminal linked to a central computer. It took merely 50 years for this technology to reach every human's pocket. Such ubiquity offers many advantages, one of which is serving as a perfect example to demonstrate real-world scenarios.
In this blog post, we will explore exactly that. We will see how Neurelo can simplify this seemingly complex process of building such an app, leveraging MongoDB and Python for this tutorial.
An enormous chunk of developer complexity lies in the layers between the application and the database, such as managing schemas, ensuring the use of the appropriate database programming interface, and writing the correct queries. Other critical layers include query controls like rate limiters and caching, health checks and database audits, planning APIs with good up-to-date documentation, setting up observability metrics, and finally, enforcing security, including a flexible access control mechanism. As we will see, Neurelo helps simplify all of this, and more.
So, what is Neurelo? A quick summary is that Neurelo instantly turns your database into APIs, which you can then use to facilitate communication with your application. But as you read above, it is much more than that – Neurelo makes working with databases way easier and removes many of the complexities developers face when building their applications with databases.
As briefly mentioned above, Neurelo offers a unified platform to help manage the complexities associated with building with databases. For instance,a significant complexity lies in managing schemas. To address this, Neurelo introduces developer-friendly schema management through the innovative concept of Schema-as-Code. By treating database schemas as code, Neurelo allows you to manage your schema changes within your existing codebase, enhancing collaboration, version control, and automation. You can then use your familiar version control systems to track changes, review diffs, and manage migrations with ease. This approach helps ensure consistency between application code and database schema.
Another major solution that Neurelo provides is auto-generation of Cloud APIs using your schemas. For developers facing challenges with serverless application reliability, latency, memory usage, and cost, Neurelo’s cloud APIs offer a compelling solution. We have optimized pooled-and-shared connection handling to ensure swift and reliable database interactions. Additionally, Neurelo’s cloud APIs support multiple databases, dynamic scaling, and secure access control, contributing to their effectiveness in any cloud environment you choose for running your apps. They adapt seamlessly to varying workloads, preventing issues like cold starts or connection limit challenges.
Not only this, but Neurelo also has built a whole ecosystem filled with wonderful tools like the “API Playground”, which is a platform that allows you to design, build, test, and collaborate on APIs with your colleagues, and the “API and Query Observability” tool to provide you with detailed query-level insights.
And wait, that’s not all. Neurelo also provides seamless introspection and migration support, a one-click solution to generate mock data, a tool to visualize your schemas, and much much more. So, without further ado, let’s see how we can build an application with Neurelo.
Here’s a figure encapsulating the process of creating your application with Neurelo. We will elaborate on this throughout our tutorial.
If you prefer a more visual learning experience, check out the videos for this tutorial here
To get started, let us navigate to Neurelo’s dashboard and create a new project. This is the primary "working" entity where everything begins within Neurelo.
To create one, navigate to and click on the 'New' button in the dashboard. A 'New Project' modal will open up, where we will fill in our basic project information, including selecting our database engine – we are going to choose MongoDB here.
Once created, Neurelo will ask you to choose a way to start building upon your project.
For new users with a data source, we encourage you to use our 'Quick Start' option. Neurelo offers users three options. If you have an existing data source and have used Neurelo before, there is an advanced option to connect this data source to Neurelo using the 'Introspection' option to get started with the schema in the database. Another option is to click on 'Empty Project' and start with building your schema/definitions with Neurelo.
For this tutorial, we will start with an 'Empty Project'. Once you click on it, you will be taken to the 'Schema' view within the 'Definitions' tab. The Schema Builder mode will be the default if you are new to Neurelo. If it’s not the mode currently set for you, you can always change it by going to the 'Mode' option on the top right, and from the dropdown, selecting 'Schema Builder'. This will open up a UI interface to easily construct our schema.
Let's now envision a schema for our real-time chat application—what would it look like? We can begin with three collections: users, conversations, and messages (Neurelo refers to these as Objects). To create objects in our schema, go to the 'New object' button on the left sidebar and enter your object name.
As you can see, Neurelo has not only created objects, but for each object, it has also initialized a default identifier. Since we are using MongoDB, this is of the type format 'ObjectId'. This identifier has a default function, auto(), associated with it, which helps automatically generate an ObjectId when you insert a new record into your collection.
With this done, let us now leverage Neurelo's 'Git Schema' functionality to commit this initial schema. To do so, click on the commit button at the top right and enter your commit message.
Next, let us further define the object by associating relevant properties with it. For example, a user can have:
A 'username' of type format 'string'
A 'meta' property that stores dynamic metadata information. For this, we can use the 'json (multiple)' type format and set Nullable as true, making it a non-required property
An 'account_creation' property of type format 'dateTime (string)'. For this, we can set the Property Default to type function with the function as now. This helps automatically generate a DateTime value with the current data source timestamp during insertions. It is also important to set the Data Source Mapping to Timestamp (another option being just Date)
More details about the types and their options can be found in our Neurelo Schema Language (NSL) guide.
Ensure that you commit your schema once you’re done editing the "users" object. Similarly, we will further define the conversations and messages objects.
For conversations, we'd like to create a property called 'type' that helps us classify the type of conversation as either 'group' or 'direct'. To do this, create a new enum by clicking the 'new enum' button on the left, then enter your enum name followed by your enum values.
Next, return to our conversation object, create a new property called 'type' of type format 'enum', and associate it with our 'conversation_type' enum.
For conversations, we will also create a property called 'usernames' of type format "array", which will store the usernames of users involved in the conversation.
Similarly, for messages, we would like to store a few properties as well, such as a "txt_hash" of type format "string." This will be useful for storing a hash of the text message, maintaining user privacy while enabling the verification of user messages. We can also store a "timestamp" of the message, similar to the "account_creation" property we saw earlier. Et voila! We now have a basic schema!
Let us now build relationships on top of this schema. What kind of relationships do we want for our chat application? We need relationships between users and conversations. One user can be part of many conversations, and similarly, one conversation can include many users. This would be a many-to-many relationship. Another relationship can be between users and messages. Each user can send multiple messages, but one message is only tied to a single user, forming a one-to-many relationship. Yet another relationship can be formed between conversations and messages. Each conversation can contain multiple messages, but one message can only be tied to a single conversation, again forming a one-to-many relationship. So, let us see how we can build these relationships.
We start by constructing the many-to-many relationship between users and messages. To do so, go to the users object and click on the "new relationship" button. This will open up Neurelo's relationship builder modal.
It will ask you to select the type of relationship; choose the many-to-many option. In the next view, we will add the details of this relationship. Neurelo should present you with three entities: source, linking, and target. Source and Target would be users and messages respectively. The linking entity will create a new object for you that stores the many-to-many relationships. This acts as our association table (or join table), meaning each record will store a conversation id and its linked user id.
Select the messages object as the target from the target object’s dropdown menu. Upon doing so, you should see the identifiers and relationship names being automatically completed for all three entities. Next, we will create two property names in our linking entity, one to store the conversation_id and another to store the user_id. When done, it looks as follows:
Similarly, one-to-many relationships between users-messages and conversations-messages can be added as follows.
And that's it! We have finished building our schema, so make sure to commit your changes.
You can click on the History icon to view all your commits. You can also check out the state of your schema at an older commit by clicking on the particular commit.
Now that we have a new schema, let's visualize it. To do so, you can use Neurelo’s ERD functionality, right from the schema view.
If you'd like to see which objects are related to others, simply click on the object whose relationships you want to explore:
Now that we have created a schema, let's explore how we can connect this schema to an empty MongoDB database and use Neurelo to automatically build API endpoints for us.
To do this, we first need to connect a data source with Neurelo. This can be achieved by navigating to the "Data Sources" tab on the left side of your project view and clicking on the new data source button. Next, enter your data source credentials. For this tutorial, we will be using a free MongoDB Atlas cluster, which we have already spun up.
Make sure to test your connection string with Neurelo’s "Test Connection" button. If all’s good, click on Submit to connect this data source with Neurelo.
Once done, you should see it in the Data Sources view for your project. You can click on "Run Health Check" to monitor whether your database is up and running. Neurelo also periodically runs health checks against the database and updates you accordingly.
With this, we now have all the components in place to start a Neurelo environment, followed by starting the runners. Think of environments in Neurelo as runtime workspaces that allow you to run your Neurelo APIs using a specific version (commit) of your schema definition against a specific data source (in this case, our "Atlas" one).
Neurelo environments are designed to naturally align with typical Software Development Life Cycles (SDLC) as applications are developed, tested, deployed, and operated. Simply put, they help manage different stages of your project, whether it's development, testing, or production. In our scenario, we will create a testing environment.
To do so, navigate to the "Environments" tab on the left sidebar of your project view and click on the new environment button. Next, fill in the environment details, which include the commit you’d like your environment to run against, and your data source/region preferences. For this tutorial, we will select the latest commit and use the same region (AWS-US West 2) as used while creating our "Atlas" data source. We have also enabled observability, which will allow us to monitor and analyze the environment's API performance.
Upon creating an environment, you’ll be greeted with the following environment view:
There is a lot going on here, so let's break it down. The APIs tab is generated and automatically updated in sync with your schema. It shows two things: a comprehensive documentation of your auto-generated API endpoints and an API reference for all the API endpoints that Neurelo has created for your schema.
Neurelo supports both REST and GraphQL APIs for your schema.
Here is an example of an API reference created for our messages object:
The APIs tab is also a hub where you can test and explore your REST and GraphQL APIs using the API playground, which we will thoroughly explore shortly. Furthermore, Neurelo also creates both OpenAPI and GraphQL specifications using your schema, which you can download by clicking on the "Specs" tab.
Let us now circle back to our chat application. Now that we have our schema, data source, and environment all set up, let's figure out how we can get some test data into our database so that we can play around with Neurelo's auto generated APIs and eventually build our chat application around it.
Neurelo provides a simple, one-click solution for this called "Data Generator." You can find this in the top right corner of the environment's view. Upon clicking it, you will be presented with an option to choose the size of your test data. For our purposes, we will select “Small” to get 10 records.
Click on start and that should be it! You can easily visualize the new data in your database using Neurelo’s Data Viewer. But before we do that, we need to start the runners.
A runner in Neurelo is the main component that executes and manages all API calls. In Neurelo's Cloud Data Access Platform, a runner is a part of every environment inside a project. Runners need to be started for the application to be able to handle API requests and process them against the configured data source within an environment. The data viewer also requires these runners to be started.
To do so, click on the "Start Runners" button on the top right of your environment’s view.
Next, navigate to the Data Viewer tab. You will be prompted to create a new API key. Make sure to copy and save this key somewhere safely, as we will be using this quite a bit in the following sections.
Once you have created your API key, you will be able to visualize your data, as follows:
Notice how the data is quite realistic! This is because Neurelo uses AI to intelligently generate mock data that aligns with your schema and its attribute contexts. In fact, even the relationship identifiers should be correctly mapped between your collections. For example, notice how conversation_ids for messages are mapped with conversation's id.
This is quite helpful when working on APIs that require relationship querying/filtering.
With our mock data in place, it is finally time to experiment with Neurelo's API endpoints.
Navigate back to the APIs tab and click on the API playground button, which should open Neurelo's API playground.
Let's warm up by running the "Find many messages" API. For this, simply go to the "Headers" tab in the API playground, and input the API key from before. When done, click on Send, and we should see all the message records in our database.
To fetch only the first record, you can add a value of 1 to the "take" parameter in the "Parameters" tab.
You can do much more complicated querying with these APIs. For instance, if we would like to fetch all the users whose username starts with the letter "a", we can do so as follows:
You can even use Neurelo's APIs to perform complex create, update, and delete queries. We will see this in further detail when we start working on our chat application using Neurelo's SDKs. As mentioned before, Neurelo’s documentation provides a thorough guide on this.
Before we jump to the SDKs, let's explore another useful aspect of the environment—the ability to monitor API performance using the "Observability" tab. Neurelo also provides latency charts for all your API endpoints, along with fine-grained usage metrics (you can access these by clicking on a specific API call in the "Slowest Operations" section).
Let us now build the core of our chat application. As mentioned before, we will be using Neurelo’s Python SDKs for this. The SDKs can be downloaded by going to the APIs tab and opening up the SDKs menu.
A gzip compressed tar archive (.tgz) SDK file should be downloaded. First, uncompress this file to get the underlying folder. Doing so, we are likely to see a directory as follows:
The downloaded SDK file will be named - neurelo-sdk-python-cmt_xxxxxx.tgz (where xxxxxx are the last 6 digits of the commit applied to the environment).
To install the necessary dependencies for this SDK, we will use the requirements.txt as follows:
If you have used Neurelo SDK’s in the past, to ensure that the older version of neurelo package is uninstalled, do a python3 -m pip uninstall neurelo and rerun your requirements.txt command from above.
Moving forward, let's create a Chat class and set up two methods to retrieve the environment configuration. This involves utilizing the Configuration and ApiClient instances as illustrated below:
Note that the basic usage requires instantiating both a Configuration and an ApiClient class. These play a role in facilitating communication with Neurelo.
For Neurelo’s Python SDKs, each object defined in our Neurelo schema has a corresponding class that can be instantiated by passing the ApiClient instance into the constructor of the object's class. For example, as our data definition contains a 'Users' object, we can instantiate the Users class by passing the ApiClient instance into its constructor, as follows:
To obtain the NEURELO_API_HOST, simply copy the host name mentioned in the Environments tab.
Additionally, you can generate a NEURELO_API_KEY by creating a new API Key from the “API Keys” tab. In our case, we have already created one while starting the Data Viewer. We can export these values using a command similar to the following in our terminal:
Now, let's expand the functionality of the Chat class. For our chat application, we will use Neurelo to log our chat activities. This requires extensive usage of the create and read APIs. Specifically, we will create a new user, new conversation, and a new message (including linking their respective relationships). To prevent duplicate entries, we will also check whether a user or conversation already exists. There is no need to check for duplicate messages, as we are simply dumping the message entries.
Here is how we can create these SDK requests with Neurelo:
When calling operations with parameters (for example, fetching all user info containing a username as XYZ), the parameters are encapsulated within specific classes. To use these parameters, we create an instance of the respective class and use it as the parameter value, as shown below.
It is important to use our models submodule to create these parameters. The resulting parameter expression, like {"username": {"equals": username}}, is standard with our REST API. More information on this can be found in our Neurelo API Reference (REST) guide.
Although you can directly make API calls with parameters using the object API class as follows: users_api.find_users(filter=<filter-condition>), we do not recommend this approach. Instead, use the models submodule to create the parameters.
And that's it! We can now leverage the Chat class in our chat application as follows:
Next, you can use the above base implementation with a web framework like FastAPI or Flask to fully build your chat application. To see an example of how to leverage FastAPI, check out the FastAPI section of our tutorial on Building Python applications with Postgres and FastAPI.
Let us say that you would like to define a rather complex query for some operations on your database. To facilitate this in a simple way, Neurelo provides Custom Queries. These essentially allow you to create Custom REST or GraphQL API endpoints.
Let us see this in action. Say we would like to find all the users who are in X conversations and have sent at least Y messages. The MQL query to do this might look like the following:
Where X and Y are the number of conversations/messages. To create an API endpoint for this raw query, we navigate back to the definitions tab and then to the “Custom Queries” view.
Next, let us create a new REST endpoint called “activeUsers”. Once done, you should be able to see the custom queries tab as follows:
Mongo custom queries requires the following format in Neurelo:
We can fit in our MQL query in the above mentioned format as follows:
Notice how we have used {{ X }} instead of X, and {{ Y }} instead of Y. This is the format used to define run-time variables in Neurelo’s custom queries. Make sure to select an appropriate type (e.g. Integer) for these variables.
We can now paste the above MQL query in the body to create this endpoint.
You can also use Neurelo’s AI Assist feature to generate this query using a natural language prompt.
You can easily test this out by opening up the “Test Query” playground as follows:
When done testing, we can commit our custom query. For MongoDB, Neurelo provides an option to directly deploy this new commit to the environment (change the environment’s commit to point at this new commit). This can be done by checking the box titled “Deploy to environment <environment_name> after commit” in the commit popup.
On committing these changes, we can even re-download our updated SDKs and reinstall the neurelo module to use this API endpoint. Using a custom query endpoint with the SDKs is as simple as:
And that’s it! If you would like to get more information on custom queries, be sure to check out our documentation on Custom APIs for Complex Queries.
If you prefer a more visual learning experience, check out our video on creating a schema, connecting it with an existing empty data source, and creating a Neurelo environment.
Similarly, here is another video that elaborates on how you can build your SDKs with MongoDB and Python:
