Queues

Queues are the primary data structure that runs in your server runtime.Jobs heavily rely on queues.

Background currenly have 4 type of queues.

• Distributed Job Queue(Redis)
• PostgreSQL Job Queue
• MongoDB Job Queue
• Job Queue(In memory)

Each queue uses its own storage class. Queues can be configured based on concurrency and processing intervals. Queues do not need to be in the same instance (except in memory queue).

Storage classes implement concurrency control mechanisms for distributed workers.

You can extend both the storage classes and queues to implement your own logic.

Main queue class is the JobQueue class. Other queues extends from the JobQueue class and override its methods if needed.

Basic Usage PostgreSQL

PostgreSQL relies on the pg package for its operations

npm install pg

import { Pool } from "pg";
import {
  PostgreSQLJobQueue,
  PostgreSQLJobStorage
} from "@backgroundjs/core";

const pool = new Pool({
  connectionString: process.env.POSTGRESQL_URL!,
});

const storage = new PostgreSQLJobStorage(pool, {
  tableName: "jobs",
});
const queue = new PostgreSQLJobQueue(storage, {
  concurrency: 2,
  maxRetries: 3,
  name: "test-queue",
  processingInterval: 1000,
});

queue.register("test-job", async (data) => {
  await new Promise((resolve) => setTimeout(resolve, 3000));
  return data;
});

await queue.add("test-job",{message:"hello from postgresql job"})

await queue.addRepeatable("test-job",{message:"hello from the postgresql repeatable"},{
    every: 1,
    unit: "seconds",
})

const scheduledTime = new Date(Date.now() + 1000 * 60);

await queue.schedule("test-job",{message:"scheduled job"},scheduledTime)