Question

Let’s combine our newfound knowledge about HTTP servers and talking to the file system and create a bridge between them: an HTTP server that allows remote access to a file system. Such a server has many uses. It allows web applications to store and share data or give a group of people shared access to a bunch of files.

When we treat files as HTTP resources, the HTTP methods GET , PUT , and DELETE can be used to read, write, and delete the files, respectively. We will interpret the path in the request as the path of the file that the request refers to.

We probably don’t want to share our whole file system, so we’ll interpret these paths as starting in the server’s working directory, which is the directory in which it was started. If I ran the server from /home/marijn/public/ (or C:\Users\marijn\public\ on Windows), then a request for /file.txt should refer to /home/marijn/public/file.txt (or C:\Users\marijn\public\file.txt ).

We’ll build the program piece by piece, using an object called methods to store the functions that handle the various HTTP methods.

var http = require("http"), fs = require("fs");

var methods = Object.create(null);

http.createServer(function(request, response) {
  function respond(code, body, type) {
    if (!type) type = "text/plain";
    response.writeHead(code, {"Content-Type": type});
    if (body && body.pipe)
      body.pipe(response);
    else
      response.end(body);
  }
  if (request.method in methods)
    methods[request.method](urlToPath(request.url),
                            respond, request);
  else
    respond(405, "Method " + request.method +
            " not allowed.");
}).listen(8000);

This starts a server that just returns 405 error responses, which is the code used to indicate that a given method isn’t handled by the server.

The respond function is passed to the functions that handle the various methods and acts as a callback to finish the request. It takes an HTTP status code, a body, and optionally a content type as arguments. If the value passed as the body is a readable stream, it will have a pipe method, which is used to forward a readable stream to a writable stream. If not, it is assumed to be either null (no body) or a string and is passed directly to the response’s end method.

To get a path from the URL in the request, the urlToPath function uses Node’s built-in "url" module to parse the URL. It takes its pathname, which will be something like /file.txt , decodes that to get rid of the %20 -style escape codes, and prefixes a single dot to produce a path relative to the current directory.

function urlToPath(url) {
  var path = require("url").parse(url).pathname;
  return "." + decodeURIComponent(path);
}

If you are worried about the security of the urlToPath function, you are right. We will return to that in the exercises.

We will set up the GET method to return a list of files when reading a directory and to return the file’s content when reading a regular file.

One tricky question is what kind of Content-Type header we should add when returning a file’s content. Since these files could be anything, our server can’t simply return the same type for all of them. But NPM can help with that. The mime package (content type indicators like text/plain are also called MIME types) knows the correct type for a huge number of file extensions.

If you run the following npm command in the directory where the server script lives, you’ll be able to use require("mime") to get access to the library:

$ npm install mime
npm http GET https://registry.npmjs.org/mime
npm http 304 https://registry.npmjs.org/mime
mime@1.2.11 node_modules/mime

When a requested file does not exist, the correct HTTP error code to return is 404. We will use fs.stat , which looks up information on a file, to find out both whether the file exists and whether it is a directory.

methods.GET = function(path, respond) {
  fs.stat(path, function(error, stats) {
    if (error && error.code == "ENOENT")
      respond(404, "File not found");
    else if (error)
      respond(500, error.toString());
    else if (stats.isDirectory())
      fs.readdir(path, function(error, files) {
        if (error)
          respond(500, error.toString());
        else
          respond(200, files.join("\n"));
      });
    else
      respond(200, fs.createReadStream(path),
              require("mime").lookup(path));
  });
};

Because it has to touch the disk and thus might take a while, fs.stat is asynchronous. When the file does not exist, fs.stat will pass an error object with a code property of "ENOENT" to its callback. It would be nice if Node defined different subtypes of Error for different types of error, but it doesn’t. Instead, it just puts obscure, Unix-inspired codes in there.

We are going to report any errors we didn’t expect with status code 500, which indicates that the problem exists in the server, as opposed to codes starting with 4 (such as 404), which refer to bad requests. There are some situations in which this is not entirely accurate, but for a small example program like this, it will have to be good enough.

The stats object returned by fs.stat tells us a number of things about a file, such as its size ( size property) and its modification date ( mtime property). Here we are interested in the question of whether it is a directory or a regular file, which the isDirectory method tells us.

We use fs.readdir to read the list of files in a directory and, in yet another callback, return it to the user. For normal files, we create a readable stream with fs.createReadStream and pass it to respond , along with the content type that the "mime" module gives us for the file’s name.

The code to handle DELETE requests is slightly simpler.

methods.DELETE = function(path, respond) {
  fs.stat(path, function(error, stats) {
    if (error && error.code == "ENOENT")
      respond(204);
    else if (error)
      respond(500, error.toString());
    else if (stats.isDirectory())
      fs.rmdir(path, respondErrorOrNothing(respond));
    else
      fs.unlink(path, respondErrorOrNothing(respond));
  });
};

You may be wondering why trying to delete a nonexistent file returns a 204 status, rather than an error. When the file that is being deleted is not there, you could say that the request’s objective is already fulfilled. The HTTP standard encourages people to make requests idempotent, which means that applying them multiple times does not produce a different result.

function respondErrorOrNothing(respond) {
  return function(error) {
    if (error)
      respond(500, error.toString());
    else
      respond(204);
  };
}

When an HTTP response does not contain any data, the status code 204 (“no content”) can be used to indicate this. Since we need to provide callbacks that either report an error or return a 204 response in a few different situations, I wrote a respondErrorOrNothing function that creates such a callback.

This is the handler for PUT requests:

methods.PUT = function(path, respond, request) {
  var outStream = fs.createWriteStream(path);
  outStream.on("error", function(error) {
    respond(500, error.toString());
  });
  outStream.on("finish", function() {
    respond(204);
  });
  request.pipe(outStream);
};

Here, we don’t need to check whether the file exists—if it does, we’ll just overwrite it. We again use pipe to move data from a readable stream to a writable one, in this case from the request to the file. If creating the stream fails, an "error" event is raised for it, which we report in our response. When the data is transferred successfully, pipe will close both streams, which will cause a "finish" event to fire on the writable stream. When that happens, we can report success to the client with a 204 response.

The full script for the server is available at eloquentjavascript.net/code/file_server.js . You can download that and run it with Node to start your own file server. And of course, you can modify and extend it to solve this chapter’s exercises or to experiment.

The command-line tool curl , widely available on Unix-like systems, can be used to make HTTP requests. The following session briefly tests our server. Note that -X is used to set the request’s method and -d is used to include a request body.

$ curl http://localhost:8000/file.txt
File not found
$ curl -X PUT -d hello http://localhost:8000/file.txt
$ curl http://localhost:8000/file.txt
hello
$ curl -X DELETE http://localhost:8000/file.txt
$ curl http://localhost:8000/file.txt
File not found

The first request for file.txt fails since the file does not exist yet. The PUT request creates the file, and behold, the next request successfully retrieves it. After deleting it with a DELETE request, the file is again missing.

This is a book about getting computers to do what you want them to do. Computers are about as common as screwdrivers today, but they contain a lot more hidden complexity and thus are harder to operate and understand. To many, they remain alien, slightly threatening things.