How to upload an in memory file data to google cloud storage using nodejs?

Issue

I am reading an image from a url and processing it. I need to upload this data to a file in cloud storage, currently i am writing the data to a file and uploading this file and then deleting this file. Is there a way i can upload the data directly to the cloud stoage?

static async uploadDataToCloudStorage(rc : RunContextServer, bucket : string, path : string, data : any, mimeVal : string | false) : Promise<string> {
  if(!mimeVal) return ''

  const extension = mime.extension(mimeVal),
        filename  = await this.getFileName(rc, bucket, extension, path),
        modPath   = (path) ? (path + '/') : '',
        res       = await fs.writeFileSync(`/tmp/${filename}.${extension}`, data, 'binary'),
        fileUrl   = await this.upload(rc, bucket, 
                            `/tmp/${filename}.${extension}`,
                            `${modPath}${filename}.${extension}`)
                   
  await fs.unlinkSync(`/tmp/${filename}.${extension}`)

  return fileUrl
}

static async upload(rc : RunContextServer, bucketName: string, filePath : string, destination : string) : Promise<string> {
  const bucket : any = cloudStorage.bucket(bucketName),
        data   : any = await bucket.upload(filePath, {destination})

  return data[0].metadata.name
}

Solution

The data can be uploaded without writing to a file by using nodes streams.

const stream     = require('stream'),
      dataStream = new stream.PassThrough(),
      gcFile     = cloudStorage.bucket(bucketName).file(fileName)

dataStream.push('content-to-upload')
dataStream.push(null)

await new Promise((resolve, reject) => {
  dataStream.pipe(gcFile.createWriteStream({
    resumable  : false,
    validation : false,
    metadata   : {'Cache-Control': 'public, max-age=31536000'}
  }))
  .on('error', (error : Error) => { 
    reject(error) 
  })
  .on('finish', () => { 
    resolve(true)
  })
})

Answered By – Akash

Answer Checked By – Mary Flores (AngularFixing Volunteer)

Leave a Reply

Your email address will not be published.