Build a Dynamic Notification Engine with NestJS, BullMQ & Redis

---

---

What is a Dynamic Notification Engine?

A dynamic notification engine is a system where notification templates, provider configurations, and API keys can be updated without modifying application code.

Instead of hardcoding notification logic, the system allows administrators to configure:

• Notification templates
• Delivery providers (email, SMS, push)
• API keys
• Trigger events

For example, if an email provider changes or an SMS API key is rotated, the update happens directly from the admin panel — no code deployment required.

---

Why Static Notification Systems Fail

Many applications implement notifications directly in code:

• Template stored in codebase
• API keys stored in environment variables
• Logic tightly coupled with business logic

The problem? Any change requires a full deployment.

Common changes that shouldn't need a deployment:

• Updating notification templates
• Switching service providers
• Rotating API credentials
• Adding new notification channels

A dynamic notification system removes this limitation entirely.

---

The Real Challenge: High Volume Notifications

Sending notifications to many users at once creates serious performance problems.

Consider this scenario — your system sends notifications to 100+ users simultaneously and the third-party provider responds slowly or throws an error:

100 users trigger notification
        ↓
API calls 3rd party provider 100 times
        ↓
Provider responds slowly
        ↓
Event loop blocks
        ↓
Entire application degrades

This leads to:

• API timeouts
• Blocked event loops
• Degraded application performance

Notifications should never be processed inside the main request cycle.

---

The Solution: Queue-Based Processing

A better architecture processes notifications using queues:

API receives request
        ↓
Job added to BullMQ queue  ← returns response immediately
        ↓
Worker consumes job async
        ↓
Worker sends notification via provider
        ↓
Success or failure logged

Queues allow the application to:

• Isolate heavy workloads from the main thread
• Process jobs asynchronously
• Retry failed notifications automatically
• Scale workers independently of the API

The main API stays fast. Workers handle the heavy lifting in the background.

---

My Implementation — BullMQ + Redis

// queue.module.ts
import { BullModule } from "@nestjs/bullmq";

@Module({
  imports: [
    BullModule.forRoot({
      connection: {
        host: process.env.REDIS_HOST,
        port: parseInt(process.env.REDIS_PORT),
      },
    }),
    BullModule.registerQueue({
      name: "notifications",
    }),
  ],
})
export class QueueModule {}

// notification.service.ts
@Injectable()
export class NotificationService {
  constructor(
    @InjectQueue("notifications") private notificationQueue: Queue,
    private configService: NotificationConfigService,
  ) {}

  async sendNotification(userId: string, eventType: string) {
    // Fetch dynamic config from DB — no hardcoding
    const config = await this.configService.getConfig(eventType);

    await this.notificationQueue.add("send", {
      userId,
      template: config.template,
      provider: config.provider,
      apiKey: config.apiKey,
    });

    // Returns immediately — worker handles delivery
    return { status: "queued" };
  }
}

// notification.processor.ts
@Processor("notifications")
export class NotificationProcessor {
  @Process("send")
  async handleNotification(job: Job) {
    const { userId, template, provider, apiKey } = job.data;

    try {
      await this.providerFactory
        .getProvider(provider, apiKey)
        .send(userId, template);
    } catch (error) {
      // BullMQ automatically retries on failure
      throw error;
    }
  }
}

---

Architecture Considerations

1. Always Use Queues for High Volume

Never send bulk notifications synchronously. Even 10 simultaneous API calls to a slow provider can degrade your application.

2. Separate Database Connection Pools

Queue workers should use separate database connections to avoid exhausting the main application pool.

// Separate pool for queue workers
const workerPool = new Pool({
  max: 5, // limited connections for workers
  connectionString: process.env.DATABASE_URL,
});

// Main app pool
const mainPool = new Pool({
  max: 20,
  connectionString: process.env.DATABASE_URL,
});

3. Centralized Redis Connection

Maintain a single Redis connection shared across all queue operations:

// redis.config.ts
export const redisConnection = {
  host: process.env.REDIS_HOST,
  port: parseInt(process.env.REDIS_PORT),
  maxRetriesPerRequest: null, // required for BullMQ
};

// Reuse this across all queues — never create multiple connections

---

Key Takeaways

---

Final Thoughts

A dynamic notification engine improves both flexibility and reliability. It allows your team to update templates, switch providers, and handle large notification volumes — all without touching the codebase.

Combined with BullMQ and Redis, it becomes a battle-tested architecture you can confidently run in production.

Have questions about queue-based architectures or notification systems? Drop a message — I'd love to help.