Centralized logging SDK with automatic batching, trace correlation, and S3 storage.

Logger SDK

The @codmir/logger package provides centralized logging for all Codmir applications with automatic batching, trace ID correlation, and S3 storage support.

@codmir/logger

v1.0.0

Centralized logging SDK with automatic batching, trace correlation, and S3 storage.

Installation

pnpm add @codmir/logger

Setup

app.module.ts
// app.module.ts
import { Module } from '@nestjs/common';
import { ConfigModule, ConfigService } from '@nestjs/config';
import { LoggerModule } from '@codmir/logger';

@Module({
  imports: [
    ConfigModule.forRoot({ isGlobal: true }),
    
    // Centralized logging
    LoggerModule.forRootAsync({
      inject: [ConfigService],
      useFactory: (config: ConfigService) => ({
        service: 'my-api',
        logsServiceUrl: config.get('LOGS_SERVICE_URL'),
        minLevel: config.get('LOG_LEVEL') || 'info',
      }),
    }),
  ],
})
export class AppModule {}

Usage

users.service.ts
// users.service.ts
import { Injectable } from '@nestjs/common';
import { LoggerService } from '@codmir/logger';

@Injectable()
export class UsersService {
  constructor(private logger: LoggerService) {}

  async createUser(data: CreateUserDto) {
    // Set trace ID for request correlation
    this.logger.generateTraceId();
    
    this.logger.info('Creating user', { email: data.email });
    
    try {
      const user = await this.prisma.user.create({ data });
      this.logger.info('User created successfully', { userId: user.id });
      return user;
    } catch (error) {
      this.logger.error('Failed to create user', { 
        error: error.message,
        email: data.email 
      });
      throw error;
    }
  }
}
📦

Auto Batching

Logs are buffered and sent in batches for efficiency

🔗

Trace Correlation

Track requests across services with trace IDs

☁️

S3 Storage

Logs stored in AWS S3, Railway, or MinIO

🎯

Level Filtering

Filter logs by debug, info, warn, or error

Environment Variables

VariableDescriptionDefault
LOGS_SERVICE_URLURL of the centralized logging service—
LOG_LEVELMinimum log level (debug, info, warn, error)info

API Reference

Logger Methods

MethodDescription
debug(message, meta?)Log debug message
info(message, meta?)Log info message
warn(message, meta?)Log warning message
error(message, meta?)Log error message
flush()Force flush buffered logs
shutdown()Flush and stop the logger

Trace Methods

MethodDescription
setTraceId(id)Set trace ID for correlation
generateTraceId()Generate and set new trace ID
getTraceId()Get current trace ID
clearTraceId()Clear trace ID

Storage Structure

Logs are stored in S3 with the following key pattern:

logs/{year}/{month}/{day}/{service}/{hour}/{batch-timestamp}.json

Example:

logs/2024/12/19/gateway/14/batch-1705330000000.json

Self-Hosting the Logs Service

The centralized logging service is available as @codmir/logs and can be self-hosted:

# Clone and start the logs service
cd apps/logs
cp .env.example .env
# Configure S3 credentials
pnpm dev

Supported S3 Backends

  • AWS S3 — Native Amazon S3
  • Railway — S3-compatible object storage
  • MinIO — Self-hosted S3-compatible storage
  • Cloudflare R2 — S3-compatible with no egress fees

Configure via environment variables:

S3_ENDPOINT=     # Leave empty for AWS S3
S3_REGION=us-east-1
S3_ACCESS_KEY_ID=your-key
S3_SECRET_ACCESS_KEY=your-secret
S3_LOGS_BUCKET=codmir-logs