Part 3. Brushing up: Logger + Environment variables

In the previous part of the tutorial, we implemented a HTTP API Web server. In this part we’ll add some more features required during development and in production: fix relative import paths to absolute, add loggers to express, customize logger level based on environment variables.

If you start from this part, you can get the sorces to start from:

$ git clone https://github.com/losikov/api-example.git
$ cd api-example
$ git checkout tags/v2.2.0
$ yarn install

TypeScript Absolute Imports Paths

There’s no a standard way how to make it work, and I often see different approaches. I use tsconfig-paths in my Node.js and ReactNative projects. Let’s install it:

$ yarn add tsconfig-paths

We need to fix our scripts in package.json to:

"start": "node -r tsconfig-paths/register ./bin/app.js",
"dev": "ts-node -r tsconfig-paths/register ./src/app.ts"

Now, we need to update our tsconfig.json. Find, uncomment, and update baseUrl and paths properties to the following values:

"baseUrl": "./",
"paths": {
  "@exmpl/*": ["src/*", "bin/*"]
},

@exmpl is our scope name; you can use your company name or whatever you need. We can update now our sources to use the scope name instead of relative paths. Make the changes in 3 files:

src/api/controllers/greeting.ts:
-import {writeJsonResponse} from '../../utils/express'
import {writeJsonResponse} from '@exmpl/utils/express'
src/api/controllers/user.ts:
-import user from '../services/user'
-import {writeJsonResponse} from '../../utils/express'
+import user from '@exmpl/api/services/user'
+import {writeJsonResponse} from '@exmpl/utils/express'
src/utils/server.ts
-import * as api from '../api/controllers'
+import * as api from '@exmpl/api/controllers'

Save all the files, and run yarn build, yarn start, yarn dev, to make sure it is still working and you get the same result as before. If you have any issues, you can download the project sources from git https://github.com/losikov/api-example. Git commit history and tags are organized based on the parts and major steps. So, you can always check out the one you are stuck on.

Express Logger

To output to console HTTP requests and responses, we make our Express server use the middleware. More info about the middleware you will find here. For our example, let’s try morgan, morganBody, and a custom one.

First, let’s install dependencies:

$ yarn add morgan @types/morgan morgan-body

For a custom logger middleware, create express_dev_logger.ts file in src/utils folder and insert the following content:

In src/utils/server.ts import new dependencies:

+ import bodyParser from 'body-parser'
  import express from 'express'
  import {OpenApiValidator} from 'express-openapi-validator'
  import {Express} from 'express-serve-static-core'
+ import morgan from 'morgan'
+ import morganBody from 'morgan-body'
  import {connector, summarise} from 'swagger-routes-express'
  import YAML from 'yamljs'
  import * as api from '@exmpl/api/controllers'
+ import {expressDevLogger} from '@exmpl/utils/express_dev_logger'

Finally, make express to use our middlewares. In the same file, after const server = express() call, where we customize error response, insert this:

If we run our backend now, and make a request to it, you’ll see an output from all three loggers: 1- the custom, 2 - morgan, 3 - morganBody:

Try it out! If you have any issues, check out the source code.

It is obvious, that we don’t need all three loggers simultaneously, and each of them can be beneficial in a specific environment or circumstances. Let’s see how can we load environment variables into our app and use them.

Node.js Environment Variables

I found dotenv-extended package satisfying all basic needs to set and use environment variables. Let’s install it and dotenv-parse-variables:

$ yarn add dotenv-extended dotenv-parse-variables @types/dotenv-parse-variables

It supports a schema, a list of all variables we need to define for our app. Create .env.schema file in config folder:

Then config/.env.defaults file with the defaults values (it doesn’t need to define all of the variables defined in the schema above):

Specific environment file should define all variables from the schema, unless they are defined in the defaults. Create config/.env.dev file where we set the values for our dev environment, when we run our app with yarn dev:

It is not a good practice to commit production environment variables to git, and they can be set in our future CI/CD pipeline, but for an educational purpose let’s create config/.env.prod which will be used by our production environment, when we run our app with yarn start:

To pass config/.env.dev and config/.env.prod to our Node.js app, update scripts in package.json to set ENV_FILE variable when we run the app for production and dev:

"start": "ENV_FILE=./config/.env.prod node -r tsconfig-paths/register ./bin/app.js",
"dev": "ENV_FILE=./config/.env.dev ts-node -r tsconfig-paths/register ./src/app.ts"

Now, let’s update the code to load the environment variables with dotenv-extended and convert them to proper types with dotenv-parse-variables. Create src/config folder and index.ts file in it with the following content:

Update server.ts file to import the config:

+ import config from '@exmpl/config'
  import {expressDevLogger} from '@exmpl/utils/express_dev_logger'

And to use the config variables when we set up our middleware:

If we run our app now, we’ll see a different output for the production and dev environments which is based on the environment variables we pass to the app:

Logger Level Customization with Winston

In different environments we need a different output from the app: maxium output during development, minimum in production, and silent during unit tests, as unit tests have their own output. Let’s integrate powerfull and fast winston logger. We start from dependencies:

$ yarn add winston

You can customize winston log levels. I like default npm. Let’s declare loggerLevel in our config in src/config/index.ts:

Define LOGGER_LEVEL in our envronment schema file, .env.schema, which we created earlier in this part. Append to the end of the file:

# see src/utils/logger.ts for a list of values
LOGGER_LEVEL=

Set the default LOGGER_LEVEL value in .env.defaults:

LOGGER_LEVEL=silent

For dev environment set debug level in .env.dev:

LOGGER_LEVEL=debug

and http for prod environment in .env.prod:

LOGGER_LEVEL=http

Create src/utils/logger.ts file, which is implemented based on winston readme:

Now, we can use our winston logger instead of a default console output. Find all “console.” in the project and update it with “logger.”. Update logger.method if needed, and import the logger in all the files you make an update:

In src/app.ts:
+ import logger from '@exmpl/utils/logger'
- console.info(`Listening on http://localhost:3000`)
+ logger.info(`Listening on http://localhost:3000`)
- console.error(`Error: ${err}`)
+ logger.error(`Error: ${err}`)

In src/utils/express_dev_logger.ts
+ import logger from '@exmpl/utils/logger'

-  console.log(`Request: ${req.method} ${req.url} at ${new Date().toUTCString()}, User-Agent: ${req.get('User-Agent')}`)
+  logger.http(`Request: ${req.method} ${req.url} at ${new Date().toUTCString()}, User-Agent: ${req.get('User-Agent')}`)
-  console.log(`Request Body: ${JSON.stringify(req.body)}`)
+  logger.http(`Request Body: ${JSON.stringify(req.body)}`)

-    console.log(`Response ${res.statusCode} ${elapsedTimeInMs.toFixed(3)} ms`)
+    logger.http(`Response ${res.statusCode} ${elapsedTimeInMs.toFixed(3)} ms`)
 
-    console.log(`Response Body: ${body}`)
+    logger.http(`Response Body: ${body}`)
In src/utils/server.ts
+ import logger from '@exmpl/utils/logger'

-  console.info(apiSummary)
+  logger.info(apiSummary)
-      console.log(`${method}: ${descriptor.map((d: any) => d.name).join(', ')}`)
+      logger.verbose(`${method}: ${descriptor.map((d: any) => d.name).join(', ')}`)

If you run the app with yarn dev now, you will see the same output as before, but with a timestamp for each line:

If you run it in prod environment with yarn build && yarn start, you will not see a verbose output:

You can download the project sources from git https://github.com/losikov/api-example. Git commit history and tags are organized based on the parts.

In the next part of the tutorial, we’ll cover our code with unit tests using jest.