NextJs Performance Mysteries: Unmasking Memory Leaks

My article is open to everyone. Please use the draft link, if you cannot read it

Memory leak is a type of resource leak where a software program is not managing the memory allocation and releasing memory when not needed in a proper way.

I have been working on Java for more than a decade and have see memory leaks in Java applications, used various tools & profiler to trace and idenitify the memory leaks, to fix it. Recently I started working on web applications on server side, interestingly I started to observe memory leak on NextJs SSR application. So I started towards tracing that. Lets start!

Common Memory Leaks in Nodejs ( Which is powering NextJS server side)

Global Variables and constants — Constants and Variables defined in global scope which can be used across requests, need proper clean up after usage.

// CODE CREATING MEMORY LEAK
// Declare the global array (not ideal, but for demonstration purposes)
const leakyArray: LeakyData[]=[]

export async function GET(req: NextRequest) {
  const data: LeakyData[] = [
    { id: 1, name: "John Doe", email: "john.doe@example.com", age: 30 },
    { id: 2, name: "Jane Smith", email: "jane.smith@example.com", age: 25 },
    { id: 3, name: "Bob Johnson", email: "bob.johnson@example.com", age: 40 },
  ];
  leakyArray.push(...data)
  console.log({ message: `Leaking memory... Array size: ${leakyArray.length}` });
  return NextResponse.json(leakyArray);
}

Closures — Javascript function which will bundle together with references to its surrounding state.

// CODE CREATING MEMORY LEAK
function createLeakyClosure() {
  let largeArray = new Array(1000000).fill(<LARGE_DATA>); // Large object

  return function() {
    console.log(largeArray.length);
  };
}

let myLeakyClosure = createLeakyClosure();
// Even after this, largeArray is kept alive in memory
// because myLeakyClosure still references it!
myLeakyClosure = null; 

Event Listeners — Functions that are called when specific event occurs, mostly these are callback functions

// CODE CREATING MEMORY LEAK
const EventEmitter = require('events');

const myEmitter = new EventEmitter();

function leakyListener() {
  const largeObject = new Array(1000000).fill('data');
  console.log('Listener called!');
  // Even if we try to clean up here, it's too late
  // because the emitter still holds a reference!
  largeObject = null; 
}
myEmitter.on('someEvent', leakyListener);
// Emit the event only once
myEmitter.emit('someEvent'); 

// ...The listener is never removed, and largeObject is still in memory

Timer s — Usage of setTimer and setInterval method in your application to do some tasks in regular time interval or once.

// CODE CREATING MEMORY LEAK
const leakyArray: LeakyData[]=[]
export async function GET(req: NextRequest) {
  const data: LeakyData[] = [
    { id: 1, name: "John Doe", email: "john.doe@example.com", age: 30 },
    { id: 2, name: "Jane Smith", email: "jane.smith@example.com", age: 25 },
    { id: 3, name: "Bob Johnson", email: "bob.johnson@example.com", age: 40 },
  ];
  setInterval(() => {
    leakyArray.push(...data);;
    hugeString += 'This is going to consume a lot of memory! '; 
    console.log('Leaking:', leakyArray.length); 
  }, 10);
  return NextResponse.json(leakyArray);
}

I/O Resource Leaks — Less common, but these are leaks arising from IO operations in your code, can me DB connections, File operations etc

// CODE CREATING MEMORY LEAK
const fs = require('fs');

function processFile(filename) {
  // Open the file for reading (this is synchronous for simplicity)
  const fd = fs.openSync(filename, 'r'); 

  // ... do some processing on the file ...

  // Oops! We forgot to close the file descriptor! 
  // fs.closeSync(fd);  
}

// Simulate processing a large number of files
for (let i = 0; i < 10000; i++) {
  processFile(`some_file_${i}.txt`);
}

console.log('Processed files!');

How to Identify these memory leaks?

When I encountered this, I used two optiosn below

Option 1: Using Node inspect in chrome browser

Yes you heard it right, be it remote apps or apps running in local you can profile the nextjs app with chrome://inspect tool.

To enable nextjs app for inspecting in chrome, run the below command. If it is for next dev, run

node --inspect ./node_modules/next/dist/bin/next dev

for built standalone nextjs application use

node --inspect server.js

Once the server is started, open chrome://inspect in a chrome tab, you will see something similar

Click inspect, and perform actions in your application to take snapshots. I used autocannon, to generate load for the app. Alternatively you can use k6 as well.

Before running the load, take a snapshot of the memory in your app. For this example there are 4 snapshots 1. After server startup 2. In the middle of the load 3.At the end of the load and 4. After cooling period of 60s

At a quick glimpse, you can see there is a memory leak. The snapshots taken has memory usage. While server started it was around 12 MB and when the load is applied and after cooling down, the memory is around 45 MB. The memory is still referenced in the app, causing the memory to maintain.

We can deep dive in to it, by comparing the snapshot, to understand which memory is retained and freed. Clicking each line item will also provide detailed stack trace.

Option 2: Usage of Clinic Js

Clinic.js is a suite of tools to help diagnose and pinpoint your Node.js performance issues. Clinic JS has 4 tools, which are really useful for various purposes

Clinic Doctor:

This tool provides automated diagnosis of your application’s performance problems. It analyzes various metrics and suggests potential solutions.

Clinic Flame:

Flame graphs visualize the call stack, revealing hot paths and bottlenecks in your code. This helps you identify which functions are consuming the most CPU time.

Clinic Bubbleprof:

Bubbleprof visualizes asynchronous operations and their relationships, making it easier to identify delays and inefficiencies in asynchronous code.

Clinic Heap Profiler:

This tool tracks memory allocation, helping you identify memory leaks and areas of excessive memory usage.

I used doctor and heapprofiler, for the memory leak we were facing. Compared to node — inspect, Clinic Js provides visual graph summary of the memory usage ( cpu, eventloop) over series of time period.

To get started lets use

clinic doctor -- node server.js

Simulate the load now with autocannon

autocannon -c 10 <URL>

heapdump and memwatch are other tools, which can be used as well. But these are outdated to me, and node — inspect covers that now.

Now with this analysis, I could able to track to one of the memory leak in my nextjs application. After fixing it, I re-ran both node-inspect and clinic doctor to see how my app behaves in terms of memory usage.

The pictures above show a better memory, but still there are leaks. Just to let you know, the major concern of memory issue which was fixed is interestingly on the circuit breaker. I am using opposum circuit breaker, which was wrapped around every outbound API call. Mistakenly, I did not close the circuit breaker post the response or circuit breaker trigger by calling the shutdown(). Adding the shutdown() improved the nextjs app memory footprint far better.

Memory Profiling in Remote NextJs apps

The options I mentioned above works well with apps running in localhost, and to debug memory issues in developer machine. But in most of the cases these issues are prevalent in production environment. How to do realtime monitoring of these leaks.

Applications in production should have robust monitoring mechanisms, tools like New Relic, Datadog and Dynatrace does this which aids in monitoring and tracking memory & resource leaks in production environments.

References

How to Remote Debug and Profile Node.js Apps? - Yonatan Kra

https://clinicjs.org/documentation/