Image returned from REST API always displays broke

Question

I am building a content management system for an art portfolio app, with React. The client will POST to the API which uses Mongoose to insert into a MongoDB. The API then queries the DB for the newly inserted image, and returns it to the client.

Here's my code to connect to MongoDB using Mongoose:

mongoose.connect('mongodb://localhost/test').then(() => 
console.log('connected to db')).catch(err => console.log(err))

mongoose.Promise = global.Promise

const db = mongoose.connection

db.on('error', console.error.bind(console, 'MongoDB connection error:'))

const Schema = mongoose.Schema;

const ImgSchema = new Schema({
  img: { data: Buffer, contentType: String }
})

const Img = mongoose.model('Img', ImgSchema)

I am using multer and fs to handle the image file. My POST endpoint looks like this:

router.post('/', upload.single('image'), (req, res) => {
  if (!req.file) {
    res.send('no file')
  } else {
    const imgItem = new Img()
    imgItem.img.data = fs.readFileSync(req.file.path)
    imgItem.contentType = 'image/png'
    imgItem
      .save()
      .then(data => 
        Img.findById(data, (err, findImg) => {
          console.log(findImg.img)
          fs.writeFileSync('api/uploads/image.png', findImg.img.data)
          res.sendFile(__dirname + '/uploads/image.png')
        }))
  } 
})

I can see in the file structure that writeFileSync is writing the image to the disk. res.sendFile grabs it and sends it down to the client.

Client side code looks like this:

handleSubmit = e => {
    e.preventDefault()
    const img = new FormData()
    img.append('image', this.state.file, this.state.file.name)
    axios
      .post('http://localhost:8000/api/gallery', img, {
        onUploadProgress: progressEvent => {
          console.log(progressEvent.loaded / progressEvent.total)
        }
      })
      .then(res => {
        console.log('responsed')
        console.log(res)
        const returnedFile = new File([res.data], 'image.png', { type: 'image/png' })
        const reader = new FileReader()
        reader.onloadend = () => {
          this.setState({ returnedFile, returned: reader.result })
        }
        reader.readAsDataURL(returnedFile)
      })
      .catch(err => console.log(err))
  }

This does successfully place both the returned file and the img data url on state. However, in my application, the image always displays broken.

Here's some screenshots:

How to fix this?

Answer 1

Avoid sending back base64 encoded images (multiple images + large files + large encoded strings = very slow performance). I'd highly recommend creating a microservice that only handles image uploads and any other image related get/post/put/delete requests. Separate it from your main application.

For example:

• I use multer to create an image buffer
• Then use sharp or fs to save the image (depending upon file type)
• Then I send the filepath to my controller to be saved to my DB
• Then, the front-end does a GET request when it tries to access: http://localhost:4000/uploads/timestamp-randomstring-originalname.fileext

In simple terms, my microservice acts like a CDN solely for images.

---

For example, a user sends a post request to http://localhost:4000/api/avatar/create with some FormData:

It first passes through some Express middlewares:

libs/middlewares.js

...
app.use(cors({credentials: true, origin: "http://localhost:3000" })) // allows receiving of cookies from front-end

app.use(morgan(`tiny`)); // logging framework

app.use(multer({
        limits: {
            fileSize: 10240000,
            files: 1,
            fields: 1
        },
        fileFilter: (req, file, next) => {
            if (!/\.(jpe?g|png|gif|bmp)$/i.test(file.originalname)) {
                req.err = `That file extension is not accepted!`
                next(null, false)
            }
            next(null, true);
        }
    }).single(`file`))

app.use(bodyParser.json()); // parses header requests (req.body)

app.use(bodyParser.urlencoded({ limit: `10mb`, extended: true })); // allows objects and arrays to be URL-encoded

...etc     

Then, hits the avatars route:

routes/avatars.js

app.post(`/api/avatar/create`, requireAuth, saveImage, create);

It then passes through some user authentication, then goes through my saveImage middleware:

services/saveImage.js

const createRandomString = require('../shared/helpers');
const fs = require("fs");
const sharp = require("sharp");
const randomString = createRandomString();

if (req.err || !req.file) {
  return res.status(500).json({ err: req.err || `Unable to locate the requested file to be saved` })
  next();
}

const filename = `${Date.now()}-${randomString}-${req.file.originalname}`;
const filepath = `uploads/${filename}`;

const setFilePath = () => { req.file.path = filepath; return next();}

(/\.(gif|bmp)$/i.test(req.file.originalname))
    ? fs.writeFile(filepath, req.file.buffer, (err) => {
            if (err) { 
              return res.status(500).json({ err: `There was a problem saving the image.`}); 
              next();
            }

            setFilePath();
        })
    : sharp(req.file.buffer).resize(256, 256).max().withoutEnlargement().toFile(filepath).then(() => setFilePath())

If the file is saved, it then sends a req.file.path to my create controller. This gets saved to my DB as a file path and as an image path (the avatarFilePath or /uploads/imagefile.ext is saved for removal purposes and the avatarURL or [http://localhost:4000]/uploads/imagefile.ext is saved and used for the front-end GET request):

controllers/avatars.js (I'm using Postgres, but you can substitute for Mongo)

create: async (req, res, done) => {
            try {
                const avatarurl = `${apiURL}/${req.file.path}`;

                await db.result("INSERT INTO avatars(userid, avatarURL, avatarFilePath) VALUES ($1, $2, $3)", [req.session.id, avatarurl, req.file.path]);

                res.status(201).json({ avatarurl });
            } catch (err) { return res.status(500).json({ err: err.toString() }); done(); 
        }

Then when the front-end tries to access the uploads folder via <img src={avatarURL} alt="image" /> or <img src="[http://localhost:4000]/uploads/imagefile.ext" alt="image" />, it gets served up by the microservice:

libs/server.js

const express = require("express");
const path = app.get("path");
const PORT = 4000;

//============================================================//
// EXPRESS SERVE AVATAR IMAGES
//============================================================//
app.use(`/uploads`, express.static(`uploads`));

//============================================================//
/* CREATE EXPRESS SERVER */
//============================================================//
app.listen(PORT);

What it looks when logging requests:

19:17:54 INSERT INTO avatars(userid, avatarURL, avatarFilePath) VALUES ('08861626-b6d0-11e8-9047-672b670fe126', 'http://localhost:4000/uploads/1536891474536-k9c7OdimjEWYXbjTIs9J4S3lh2ldrzV8-android.png', 'uploads/1536891474536-k9c7OdimjEWYXbjTIs9J4S3lh2ldrzV8-android.png')

POST /api/avatar/create 201 109 - 61.614 ms

GET /uploads/1536891474536-k9c7OdimjEWYXbjTIs9J4S3lh2ldrzV8-android.png 200 3027 - 3.877 ms

What the user sees upon successful GET request: