Introducing NeDB

NeDB Overview:

NeDB is an embedded NoSQL database with an API that resembles MongoDB. Unlike the previous lab where data was stored in memory, in this lab, data will be persisted in a database.
- Setting Up Databases:
  
  You will create two databases, messages and users, to store relevant data.
```
const messages = new Datastore({ filename: 'db/messages.db', autoload: true });
const users = new Datastore({ filename: 'db/users.db', autoload: true });
```
Task:

Modify the server-side routing functions so that messages and users are stored in these databases. This will allow the application to persist data across restarts and handle larger data volumes.

Pagination

Problem:

As your application grows, storing and retrieving large collections (e.g., messages) can become inefficient. Fetching all messages at once can slow down the app.
Solution:

Implement pagination by fetching only the most recent n messages, in this case, 5. You can achieve this by sorting the messages by their creation timestamp.
- Adding Timestamps:
  
  NeDB allows you to automatically add timestamps to new entries by using the timestampData option.
```
const messages = new Datastore({ filename: 'db/messages.db', autoload: true, timestampData: true });
```
- Retrieving Recent Messages:
  
  You can retrieve the latest n messages using the following query:
```
messages.find({}).sort({ createdAt: -1 }).limit(5).exec((err, data) => { });
```
Task:

Update the routing function to retrieve the 5 latest messages and return them to the frontend.

Uploading and Storing Files

File Upload Problem:

In this part of the lab, the goal is to handle file uploads properly, specifically for user profile images. Initially, when a file is uploaded, only the filename is sent, which is insufficient because the server does not have access to the file content.
Solution:

The form must be modified to send the file content in the body of the HTTP request, using the multipart/form-data encoding.
- Frontend Modification:
  
  Update the form to use multipart/form-data so that the image file can be sent properly:
```
<form id="signup" action='/api/users/' method="POST" enctype="multipart/form-data">
```
- Backend Handling with Multer:
  
  Use multer, a middleware for handling file uploads, to store the image on the server.
```
import multer from "multer";
const upload = multer({ dest: 'uploads/' });

app.post('/api/users/', upload.single('picture'), (req, res) => {
  // Process file and store it
  res.redirect('/');
});
```

Serving the Image:

Rather than serving the uploaded images from the static directory, a better approach is to dynamically serve the images via a route.

app.get('/api/users/:username/profile/picture/', (req, res) => {
  const profile = /* retrieve the user's profile */;
  res.setHeader('Content-Type', profile.mimetype);
  res.sendFile(profile.path);
});