How to Build a URL Shortener With Node.js and MongoDB

How to Build a URL Shortener With Node.js and MongoDB

If you've ever embedded a link, you've probably used an app liky bit.ly or goo.gl. In this tutorial, we'll learn how to make our own version of these popular web apps!

In this post, we’ll show you how to build a URL shortening service like bit.ly or goo.gl using Express.js (Node.js) and MongoDB. Here’s a demo of the final product we’ll be building through our MongoDB hosting platform.

How Does a URL Shortener Work?

At a very high level, the URL shortener works by taking an entered URL and creating a relatively shortened version simplified into an easy to share format. The shortened hash will be generated by base-encoding an auto-incremented counter and creates a minimum three-character hash that increases as the number of stored URLs go up.

When the shortened version of the URL is visited, the service will decode the hash to fetch the original URL stored in MongoDB and then redirect your user to it.

Getting Started

Here’s a list of the technologies we’ll use to build the URL shortener in this tutorial:

Express.js (Node.js backend)

MongoDB (storing URLs)

HTML, CSS, JavaScript (front-end)

URL Shortener Tutorial

Setup the MongoDB Database Structure

  1. Let’s start by creating a Shared MongoDB Cluster on ScaleGrid. This is the easiest way to create a quick cluster, but you can also install MongoDB on your machine and get started there.Once the cluster is created, you’ll be provided with a connection string that can be copied with a single click from your Cluster Details page. We’ll need this string to connect to the cluster from our application. Remember, never share your connection string with anyone.We’ll need two collections for the URL shortener:

Collection 1

Collection 2

Set Up the Express.js Backend

  1. Here’s a list of dependencies required to set up our Node.js backend:
{
  "name": "sg-url-shortener",
  "version": "1.0.0",
  "description": "A simple URL shortener built with Node.js and MongoDB",
  "dependencies": {
    "atob": "^2.0.3",
    "body-parser": "^1.15.2",
    "btoa": "^1.1.2",
    "dotenv": "^4.0.0",
    "express": "^4.10.2",
    "mongoose": "^4.13.7"
  },
  "main": "index.js",
  "scripts": {
    "start": "node index.js"
  },
  "engines": {
    "node": "4.8.4"
  }
}
  1. Run npm install to install all the required dependencies. Once all of our dependencies have been set up, we need to connect to our Shared MongoDB Cluster. Create a .env file in the root of the project and add the connection string to it. You can get the connection string from your Cluster Details page under the Overview tab on the ScaleGrid Console.
connectionString=mongodb://user:[email protected]
  1. Before we start writing code, it’s a good practice to visualize the app flow so we have a good understanding of how the shortening process will work. Here’s a diagram showing the process of URL shortening:Here’s a diagram showing the process of redirection when a shortened URL is visited:Now that we’ve visualized the entire process, it’s time to translate the above flowcharts into code.

Initializing the Application

  1. Before we start writing business logic, we need to initialize our application with our node modules and set up a server. Load .env files only in dev mode. Since the demo application is hosted on Heroku, an environment variable has been created from the Heroku dashboard that already contains the connection string there:
if(process.env.NODE_ENV !== 'production') {
    require('dotenv').load();
}
  1. Application initialization, server, and middleware setup. Note that we are also getting the connection string from the environment variable:
var express = require('express'),
    bodyParser = require('body-parser'),
    app = express(),
    http = require('http').Server(app),
    mongoose = require('mongoose'),
    btoa = require('btoa'),
    atob = require('atob'),
    promise,
    connectionString = process.env.connectionString,
    port = process.env.PORT || 8080;

http.listen(port, function() {
    console.log('Server Started. Listening on *:' + port);
});

app.use(express.static('public'));
app.use(bodyParser.urlencoded({
    extended: true
}));
  1. Base-route for loading up the front-end of our app:
app.get('/', function(req, res) {
    res.sendFile('views/index.html', {
        root: __dirname
    });
});

Storing URLs in MongoDB

  1. Let’s start by creating the collection schemas for storing data. As discussed above, we need two collections: one for storing the auto-incremented counter and the other for storing the URLs.
var countersSchema = new mongoose.Schema({
    _id: { type: String, required: true },
    count: { type: Number, default: 0 }
});

var Counter = mongoose.model('Counter', countersSchema);

var urlSchema = new mongoose.Schema({
    _id: {type: Number},
    url: '',
    created_at: ''
});

urlSchema.pre('save', function(next) {
    console.log('running pre-save');
    var doc = this;
    Counter.findByIdAndUpdate({ _id: 'url_count' }, { $inc: { count: 1 } }, function(err, counter) {
        if(err) return next(err);
        console.log(counter);
        console.log(counter.count);
        doc._id = counter.count;
        doc.created_at = new Date();
        console.log(doc);
        next();
    });
});

var URL = mongoose.model('URL', urlSchema);
  1. The above code creates the two collections and sets up our database for storing these collections. We’re also using a pre-save hook for the URL schema since we need to auto-increment the counter and log the date and time at which the URL was created. Next, we need to make sure that we start our application fresh and all previous entries are deleted. Once we reset, we’ll initialize our counter with a starting value of 10,000 to set up the URL shortening process. You may start with any value. This was chosen at random and will auto-increment by a value of one.
promise = mongoose.connect(connectionString, {
    useMongoClient: true
});

promise.then(function(db) {
    console.log('connected!');
    URL.remove({}, function() {
        console.log('URL collection removed');
    })
    Counter.remove({}, function() {
        console.log('Counter collection removed');
        var counter = new Counter({_id: 'url_count', count: 10000});
        counter.save(function(err) {
            if(err) return console.error(err);
            console.log('counter inserted');
        });
    });
});
  1. Our application is now ready to start accepting and shortening URLs! Let’s create a POST API that our front-end will use to send the URL:
app.post('/shorten', function(req, res, next) {
    console.log(req.body.url);
    var urlData = req.body.url;
    URL.findOne({url: urlData}, function(err, doc) {
        if(doc) {
            console.log('entry found in db');
            res.send({
                url: urlData,
                hash: btoa(doc._id),
                status: 200,
                statusTxt: 'OK'
            });
        } else {
            console.log('entry NOT found in db, saving new');
            var url = new URL({
                url: urlData
            });
            url.save(function(err) {
                if(err) return console.error(err);
                res.send({
                    url: urlData,
                    hash: btoa(url._id),
                    status: 200,
                    statusTxt: 'OK'
                });
            });
        }
    });
});
  1. As outlined in the flow diagram, once a valid URL is received, we check for its existence in the database. If found, we decode the corresponding _id field and return the hash back. Our front-end constructs the shortened URL and presents it to the user for redirection. If no URL is found, we save a new document in the collection. Remember, a pre-save step is run everytime the URL is saved. This will auto-increment the counter and log the current date and time. After the document is added, we send the hash to our front-end which constructs the shortened URL and presents it to the user for redirection.

Redirecting Users

  1. We’re almost done! Once our shortened URLs have been created, we need a way to redirect the user when a shortened URL is visited.
app.get('/:hash', function(req, res) {
    var baseid = req.params.hash;
    var id = atob(baseid);
    URL.findOne({ _id: id }, function(err, doc) {
        if(doc) {
            res.redirect(doc.url);
        } else {
            res.redirect('/');
        }
    });
});
  1. The above code looks for a hash in the shortened URL, base64 decodes it, checks if that ID is present in the collection, and redirects the user accordingly. If no ID is found, the user is redirected to the homepage of the URL shortener. For front-end code, please check out the GitHub repository mentioned at the end of this post. It’s essentially a text box field with a button to send the URL to the back-end and is out of the scope of this article.

More URL Shortener Enhancements

And we’re done! We have a bare-bones URL shortener that can be used internally to simplify your links. If you’d like to add more bells and whistles, here is a list of things you can additionally implement:

The entire code is available here: ScaleGrid URL Shortener Code Samples - GitHub

A demo application is hosted on Heroku: ScaleGrid URL Shortener Demo.

Originally published by Kunal Nagar at https://dzone.com

Learn More

☞ The Complete Node.js Developer Course (2nd Edition)

☞ Learn and Understand NodeJS

☞ Node JS: Advanced Concepts

☞ GraphQL: Learning GraphQL with Node.Js

☞ Angular (Angular 2+) & NodeJS - The MEAN Stack Guide

☞ Beginner Full Stack Web Development: HTML, CSS, React & Node

☞ Node with React: Fullstack Web Development

☞ MERN Stack Front To Back: Full Stack React, Redux & Node.js