Part 5. MongoDB with mongoose

San Diego Zoo, Freedom Out There

In this part of the tutorial, we’ll learn how to work with MongoDB using mongoose. We’ll follow the best practices and develop our backend together with unit tests using jest. In the end, we’ll run the server with a real MongoDB using docker, and try the API calls using curl, to make sure it works. We’ll work on a single API call to register/create a user, but we’ll add more code now which we’ll use in the next parts.

$ git clone https://github.com/losikov/api-example.git
$ cd api-example
$ git checkout tags/v4.0.0
$ yarn install

Mongoose, Initial Setup

Let’s install dependencies first:

• mongoose — mongodb object modeling library for node.js;
• bcrypt — library to store and verify passwords securely;
• mongodb-memory-server — lightweight mongodb server for unit tests (can be installed as dev dependency);
• faker — fake data generator for unit tests.

$ yarn add mongoose @types/mongoose bcrypt @types/bcrypt mongodb-memory-server @types/mongodb-memory-server 
$ yarn add -D faker @types/faker

We’ll need MongoDB configuration variables for our dev, prod, test environments. Let’s update our .env.* files.

To the schema file, config/.env.schema, add:

MONGO_URL=
MONGO_CREATE_INDEX=
MONGO_AUTO_INDEX=

To the file with default values, config/.env.defaults, add:

MONGO_CREATE_INDEX=true

For dev environment, file config/.env.dev, add:

MONGO_URL=mongodb://localhost/exmpl
MONGO_AUTO_INDEX=true

For prod environments, file config/.env.prod, add:

MONGO_URL=mongodb://localhost/exmpl
MONGO_AUTO_INDEX=false

You can read more about auto index.

Finally, for test environment, file config/.env.test, add:

MONGO_URL=inmemory
MONGO_AUTO_INDEX=true

For the convenience, to read the variables easily, update src/config/index.ts file. Add mongo section to Config interface, and to config constant:

We are ready to create a connection to MongoDB. Create db.ts file in src/utils folder, with the following content:

As you see, we create a singleton which establishes a connection to MongoDB, inmemory or usual db. When we work with mongoose models, it will utilize the connection. Please pay attention to mongoose connection options in lines 12–21. You might need to adjust them to tune up for your environment. Also, check the code related to inmemory db, lines 4, 37–41, 72–74. Ideally, we could add mongodb-memory-server as a dev dependency and split the code for test environment, but for the sake of example simplicity, I keep it in a single file.

Now, we can update our src/app.ts to establish the connection with the MongoDB, before we create an http server:

In each unit test, where we need to work with db, we’ll open and close the connections as well.

We are ready to develop the model.

User Model

Let’s implement the DB model for a user, with email, password, name, and created properties. email, password, name must be passed, and created will be filled in automatically. Create src/api/models folder and user.ts file in it with the following content:

Please investigate the source file:

Line 22 — we utilize validator.isEmail; we won’t be able to save non-email value to the DB.

Line 24 — created will initialize automatically with Date.now value.

Line 20, 22, 24 — we make password, email, and name required values.

Line 26 — email will be unique ignoring the case.

Line 28–44 — when we save a user, we don’t save password in readable format, its password is hashed using bcrypt; and

Line 56–65 — comparePassword is used to check if password is valid, required for login operation.

Line 46–54 — user.toJSON() will return nice output without redundant properties.

To develop and test the user model, create src/api/models/__tests__ folder and user.ts file in it, with the following content:

I don’t develop the model from the beginning to the end, and develop the unit tests afterward. I write the tests along with the model code, step by step. Initially, to save and update user, then for the password, and finally to get a json, which I added just to show more capabilities. Also, I run the tests with the --coverage flag, to check if I missed something:

Pay attention to 3 groups of tests: saving user, password saving & comparing, toJSON method. My goal was to cover 100% of the code base. For production development, I’d add more extensive tests to check all corner cases and validate all output.

The model unit tests is a code base which I reuse to work on the next part, the User Service. And, I’m 100% sure, that I have the working code already.

User Service

We already have auth function in src/api/services/user.ts. Let’s add createUser function with 3 arguments: email, password and name.

As you see, it is pretty straightforward. In the unit tests we need to cover only 3 cases: successful user creation (line 16 in the code snippet above), user duplicated or already exists (lines 19-20), and any error thrown by user model (line 22–23). Let’s update src/api/services/__tests__/user.ts to do it. Add required imports, db.open() in beforeAll and db.close() in afterAll, and describe(‘createUser’ …) function which covers 3 cases we’ve just discussed:

Now, we can run the test:

If we run it with --coverage flag, it will show us, that src/api/services/user.ts is covered 100%.

We are ready to go to the final step, User Controller to connect HTTP API request with the service.

User Controller

To make express process POST /api/v1/user requests, let’s update config/openapi.yml. Always use online Swagger Editor for it, or run it locally using docker with ./scripts/open_swagger_editor.sh script which we created in one of the previous parts.

Insert new user tag in the tags section, /user section in the paths section, and FailResponse in the components/schemas section:

Don’t forget that we need to insert the content of the config/openapi.yml file into the Swagger Editor, and after you are done with the modifications, paste it back to the config/openapi.yml file and save it.

The line operationId: createUser specifies a controller function which will be called when POST /api/v1/user comes. Let’s implement it. Add createUser function to src/api/controllers/users.ts file:

Create unit tests file src/api/controllers/__tests__/users.ts:

We have only 3 tests:

• ‘should return 201 & valid response for valid user’ — basic functionality which calls writeJsonResponse(res, 201, resp) in line 23 of the previous code snippet controllers/user.ts.
• ‘should return 409 & valid response for duplicated user’ — we create a user with the same email twice, and force the controller to call writeJsonResponse(res, 409, resp) in line 18 of the previous code snippet controllers/user.ts.
• ‘should return 409 & valid response for duplicated user’ — we verify express OpenAPI validator, and it doesn’t get to the createUser controller at all, and processed with the following code in src/utils/server.ts:

Line 20 and 26–29 of the controllers/user.ts code snippet above is still not called. Let’s check it with the yarn test:unit --coverage:

We can mock UserService.createUser the same as we did for UserService.auth in src/api/controllers/__tests__/user_failure.ts. Add faker import and new describe('createUser failure', …) function to it:

Now, we are fine with the unit test coverage:

At this stage, we are ready to push and deploy the code. The only thing which I usually do before is to run yarn build, as ts-jest might have different TypeScript build settings in comarison with tsc.

Run dev environment with MongoDB docker

Sometimes it happens that something doesn’t work and unit tests don’t help you to find the problem, as it can be the configuration issue, etc. For this purpose, you can update config/.env.dev to use inmemory db, but also, it might be useful to run our server with a real MongoDB. The easiest way is to run MongoDB using docker. Let’s create a script for it:

$ touch scripts/run_dev_dbs.sh
$ chmod +x scripts/run_dev_dbs.sh

Insert the following content into the file:

For convenience, there’re 3 options in the script: to kill (-k) the running db, to cleanup (-c) the db (db files are stored in ../docker/mongodb), and run (-r) it:

Run mongo

Now, we can run our Node.js server, and try to send POST /api/v1/user to the server with a curl command :

$ curl -d '{"email": "test@test.com", "password": "Password1", "name": "Test Name"}' -H 'Content-Type: application/json' http://192.168.1.108:3000/api/v1/user

Node.js+MongoDB vs curl

Please pay attention, that I run MongoDB with docker on a local machine (192.168.1.4) while I run Node.js backend on a virtual Ubuntu (for clean environment). A simplicity to setup an environment using .env conf files, covered in the previous part, is in place.

To browse MongoDB I use a free Robo 3T. Here’s a user we’ve just created with a curl request:

If you have any problems with the code, you can download the project sources from git https://github.com/losikov/api-example. Git commits history and tags are organized based on the parts.

In this part, we’ve learned how to work with MongoDB with mongoose in Node.js, how to develop a new API for a Node.js express server with a help of unit tests, and how to run dev server with a MongoDB using docker.

In the next part of the tutorial, we’ll check how to use JSON Web Token (JWT) for authentication.