Chaining Image & Video Transformations with Sharp and FFmpeg

Modern applications rely heavily on images and videos, including user uploads, product photos, social content, and dynamically generated media assets. The challenge is not limited to storing these files. Efficient processing is also required, such as cropping, resizing, converting formats, generating thumbnails, extracting frames, optimizing for the web, and preparing assets for machine learning workflows.

Traditional workflows write temporary files to disk at each step. This slows down processing and complicates deployment, particularly in serverless environments, containerized systems, and high-throughput media APIs. Disk I/O becomes a bottleneck, and managing temporary files often introduces unnecessary complexity and recurring bugs.

In-memory media processing solves these issues. By using buffers and streams instead of the filesystem, performance improves significantly, latency decreases, and the overall architecture becomes more scalable and portable.

In the Node.js ecosystem, two tools are central to this workflow:

• Sharp for fast, low-memory image manipulation
• FFmpeg for powerful video (and audio) processing

Each tool performs well independently, but together they create a flexible and high-performance foundation for modern media pipelines. Both libraries support fully in-memory operation, which makes it possible to chain transformations without touching disk.

The result is a clean and efficient workflow in which an image or video passes through consecutive transformations, moving from one buffer to the next until the final output is produced.

This article explains how Sharp and FFmpeg can be combined in Node.js to build in-memory transformation pipelines. It covers loading media as buffers, streaming data in and out of both tools, and chaining multiple operations for images and videos.

Why Combine Sharp and FFmpeg?

Sharp and FFmpeg address different aspects of media processing, yet many production environments require the strengths of both. Sharp is optimized for fast, memory-efficient image manipulation, while FFmpeg is the industry-standard tool for video and audio processing. When combined, the two form a unified workflow capable of supporting virtually any media transformation required in modern applications.

Sharp: The Image Specialist 

Sharp is well-suited for tasks such as:

• Resizing and cropping
• Rotating or flipping
• Converting to formats like PNG, WebP, AVIF, JPEG
• Compressing for performance
• Handling large images with minimal memory overhead

It is extremely fast, built on top of the libvips library, and designed for high-throughput environments.

FFmpeg: The Multimedia Powerhouse

FFmpeg is used for:

• Video transcoding (mp4 to webm, h264 to h265, and other formats)
• Audio extraction or conversion
• Scaling or cropping video
• Reading and writing frames
• Generating GIFs
• Extracting thumbnails
• Converting image sequences to video

In short, FFmpeg is capable of processing nearly any video or audio file.

Where They Intersect

Although their domains differ, a natural bridge exists between the two tools:

• FFmpeg can extract frames from videos, but its image-processing capabilities are limited.
• Sharp can perform advanced image transformations, but it can’t work with videos directly.

Combined, they provide the strengths required for both image and video workflows: 

This pairing makes it possible to support scenarios such as:

• Generating video thumbnails, then cropping/resizing them with Sharp
• Extracting multiple frames, running them through Sharp, then reassembling them
• Normalizing all uploaded media (images + video) through a shared pipeline
• Building a media API that applies the same transformation style everywhere

In-Memory Synergy

The real advantages appear when all operations run in memory: 

• FFmpeg streams output directly into a buffer
• Sharp consumes that buffer, applies transformations, and produces a new buffer
• No temporary files
• No disk I/O
• Minimal latency

By combining FFmpeg’s video capabilities with Sharp’s image processing tools, the result is a flexible, zero-disk, high-performance media engine suitable for demanding pipelines.

Working With Sharp

Sharp is a widely used library in Node.js for high-performance image manipulation. It is built on top of libvips, a highly efficient image processing library that uses significantly less memory than tools such as ImageMagick or GraphicsMagick. Sharp also integrates seamlessly with buffers, which makes it well suited for in-memory processing pipelines.

Loading an Image from a Buffer

Sharp accepts a Buffer as input without requiring a file path. This allows images to be loaded directly from memory. 

1const sharp = require("sharp");
2
3const transformed = await sharp(inputBuffer)
4  .resize(800)
5  .toBuffer();
Copy

Each transformation produces a new buffer that can be used in the next stage of the pipeline.

Chaining Transformations

Sharp’s API is designed for chaining, making it easy to apply multiple operations in a clear and declarative manner. 

1const outputBuffer = await sharp(inputBuffer)
2  .resize(1200, 800, { fit: "cover" })
3  .rotate(90)
4  .extract({ left: 100, top: 50, width: 500, height: 300 }) // crop
5  .toFormat("webp")
6  .toBuffer();
Copy

This single chain:

1. Resizes the image
2. Rotates it
3. Crops a region
4. Converts it to WebP
5. Outputs the final result as a buffer

Sharp optimizes these operations internally to keep the pipeline efficient and to avoid unnecessary decoding or encoding steps.

Creating a Reusable Sharp Pipeline Function

In a buffer-driven workflow, utility functions are often created to accept a buffer and return a transformed buffer. 

1async function processImage(buffer) {
2  return sharp(buffer)
3    .resize(600)
4    .toFormat("png")
5    .toBuffer();
6}
Copy

This structure makes it straightforward to compose multiple transformations and build consistent image-processing sequences.

1let transformedBuffer = await processImage(originalImageBuffer);
2transformedBuffer = await anotherTransform(transformedBuffer);
Copy

Streaming Instead of Buffers

Buffers are simple and convenient for small or medium-sized images, but they require the entire file to be loaded into memory at once. This is manageable for a 200 KB PNG, but becomes difficult when processing large assets or supporting high-throughput workloads. 

In these situations, Sharp’s streaming mode becomes essential. Streaming processes data incrementally, which keeps memory usage low and maintains strong performance. This approach is far better suited for large files or environments where memory efficiency is a priority.

1const transformStream = sharp().resize(800).toFormat("jpeg");
2inputStream.pipe(transformStream).pipe(outputStream);
Copy

Working with FFmpeg

FFmpeg is widely regarded as the standard for video and audio processing. It can read, transform, and encode nearly any media format. Many examples rely on file-based workflows such as input.mp4 to output.mp4, but high-performance pipelines often require a different approach. 

FFmpeg can stream data through stdin and stdout, allowing it to receive and produce raw data directly in memory. This makes it possible to integrate FFmpeg into a buffer-driven transformation chain in the same way Sharp is used.

Streaming FFmpeg Input from a Buffer

To run FFmpeg entirely in memory, a buffer can be written to its stdin stream while the transformed output is captured from stdout. The minimal example below uses child_process.spawn to illustrate this pattern.

1const { spawn } = require("child_process");
2
3async function runFFmpeg(input: Buffer, ffmpegArgs: string[]): Buffer {
4  return new Promise((resolve, reject) => {
5    const ffmpeg = spawn("ffmpeg", [
6      "-i", "pipe:0", // read from stdin
7      ...ffmpegArgs,
8      "pipe:1"        // write to stdout
9    ]);
10    const data = [];
11    ffmpeg.stdout.on("data", chunk => data.push(chunk));
12    ffmpeg.on("close", () => resolve(Buffer.concat(data)));
13    ffmpeg.on("error", reject);
14    ffmpeg.stdin.write(input); // Feed the input buffer
15    ffmpeg.stdin.end();
16  });
17}
Copy

Once structured this way, FFmpeg effectively becomes a pure function that receives an input buffer, processes it, and returns an output buffer. This aligns well with the requirements of a buffer-driven pipeline.

Example: Extracting a Frame as an Image

A common use case is thumbnail generation. FFmpeg can extract a single frame with a straightforward command:

1const frameBuffer = await runFFmpeg(videoBuffer, [
2  "-ss", "00:00:01.000", // first frame time code
3  "-frames:v", "1",
4  "-f", "image2pipe",
5  "-vcodec", "mjpeg"
6]);
Copy

The resulting buffer contains a JPEG image that can then be processed further with Sharp.

Example: Cropping or Resizing

FFmpeg provides robust tools for cropping video content:

1const croppedVideo = await runFFmpeg(videoBuffer, [
2  "-vf", "crop=800:600",
3  "-f", "mp4"
4]);
Copy

Or for resizing:

1const resizedVideo = await runFFmpeg(videoBuffer, [
2  "-vf", "scale=320:240",
3  "-f", "mp4"
4]);
Copy

The output buffer produced by these operations contains the cropped or resized MP4 video.

Working with Streams

FFmpeg is not limited to buffer-based workflows. It is also designed to operate efficiently with streams, which is essential when processing large video files. Buffers are convenient for small or medium assets, but loading a 500 MB or multi-gigabyte video into memory can overwhelm a service. 

Streams address this by sending data to FFmpeg in small chunks through stdin, allowing processing to begin immediately without loading the entire file into RAM. This makes FFmpeg’s streaming mode well suited for high-performance pipelines, large file uploads, live transcoding, and environments that require strong scalability and low memory usage.

1const { spawn } = require("child_process");
2
3function runFFmpegStream(input: Readable, ffmpegArgs: string[]): Readable {
4  const ffmpeg = spawn("ffmpeg", [
5    "-i", "pipe:0", // read from stdin
6    ...ffmpegArgs,
7    "pipe:1"        // output to stdout
8  ]);
9
10  input.pipe(ffmpeg.stdin); // Pipe input into FFmpeg
11  return ffmpeg.stdout; // return a readable stream with transformed data
12}
Copy

Conclusion

Modern media-processing workflows require more than basic file manipulation. They demand speed, scalability, and efficient handling of both images and videos. Combining Sharp and FFmpeg within a fully in-memory, buffer-based or streaming architecture produces pipelines that remain fast and flexible, even when processing large files. This approach avoids disk access and prevents unnecessary strain on system resources.

Sharp delivers strong performance and quality for image transformations. FFmpeg provides extensive capabilities for video and audio processing. When streams are used to move data from one stage to the next, the two tools form a unified and extensible workflow that supports use cases ranging from simple thumbnails to complex video editing tasks.

In production environments, these transformation pipelines may be placed behind a CDN layer that determines when a transformation should run, how requests are normalized, and whether cached assets can be served directly. The next article explores how CloudFront request events and edge logic can be used to orchestrate these pipelines efficiently, pushing decision-making to the edge before requests ever reach the origin.