Best Practices

This guide outlines recommended practices for building secure, efficient, and maintainable integrations with the Synthreo Builder API. Following these guidelines will help ensure your applications are robust and provide a great user experience.

Security Best Practices

Credential Management

Never expose credentials in client-side code. Store your email, password, and user ID securely using environment variables or a dedicated secret management service.

// ❌ Bad: Hardcoded credentials
const credentials = {
  email: "user@example.com",
  password: "mypassword123",
  userId: 12345
};

// ✅ Good: Environment variables
const credentials = {
  email: process.env.SYNTHREO_API_EMAIL,
  password: process.env.SYNTHREO_API_PASSWORD,
  userId: parseInt(process.env.SYNTHREO_USER_ID)
};

Token Security

Implement secure token storage and refresh logic. JWT tokens expire after 24 hours and must be refreshed by re-authenticating.

class TokenManager {
  constructor() {
    this.token = null;
    this.tokenExpiry = null;
  }

  isTokenValid() {
    if (!this.token || !this.tokenExpiry) return false;
    // Refresh token 5 minutes before expiry (24-hour lifetime)
    return Date.now() < (this.tokenExpiry - 5 * 60 * 1000);
  }

  async getValidToken() {
    if (!this.isTokenValid()) {
      await this.refreshToken();
    }
    return this.token;
  }

  async refreshToken() {
    // Note: No refresh endpoint exists - must re-authenticate
    const response = await this.authenticate();
    this.token = response.token;
    // JWT tokens have 24-hour expiry
    this.tokenExpiry = Date.now() + (24 * 60 * 60 * 1000);
  }

  // Critical: Never log or expose tokens
  logSafeInfo() {
    console.log('Token status:', {
      hasToken: !!this.token,
      isValid: this.isTokenValid(),
      expiresIn: this.tokenExpiry ? Math.floor((this.tokenExpiry - Date.now()) / 1000) : null
    });
  }
}

HTTPS and Data Protection

Always use HTTPS for all API communications to ensure data is encrypted in transit.

Validate and sanitize input data before sending it to the API to prevent injection attacks.

function sanitizeUserInput(input) {
  if (typeof input !== 'string') {
    throw new Error('Input must be a string');
  }
  
  // Remove potentially harmful characters
  return input.replace(/[<>\"']/g, '').trim();
}

Asynchronous Operations Best Practices

Job Execution and Polling

Choose the right execution method based on expected operation duration:

• Synchronous (/Execute): Quick operations under 30 seconds
• Asynchronous (/ExecuteAsJob): Long-running operations (5-15 minutes typical)

Implement intelligent polling strategies:

class JobManager {
  constructor() {
    this.activeJobs = new Map();
  }

  async executeAndPoll(diagramId, payload, options = {}) {
    const {
      initialInterval = 30000,    // 30 seconds initial polling
      maxInterval = 300000,       // 5 minutes maximum interval
      timeout = 3600000,          // 1 hour timeout
      backoffMultiplier = 1.5     // Gradual increase in polling interval
    } = options;

    // Start the job
    const jobResponse = await this.startJob(diagramId, payload);
    const jobId = jobResponse.job.id;
    
    console.log(`Job ${jobId} started, polling every ${initialInterval/1000} seconds`);
    
    let currentInterval = initialInterval;
    const startTime = Date.now();
    
    while (Date.now() - startTime < timeout) {
      await new Promise(resolve => setTimeout(resolve, currentInterval));
      
      try {
        const statusResponse = await this.checkJobStatus(jobId);
        
        if (statusResponse.status === 202) {
          // Job still running - increase polling interval gradually
          currentInterval = Math.min(currentInterval * backoffMultiplier, maxInterval);
          console.log(`Job ${jobId} still running, next check in ${currentInterval/1000} seconds`);
          continue;
        }
        
        if (statusResponse.status === 200) {
          console.log(`Job ${jobId} completed successfully`);
          return statusResponse.data;
        }
        
        throw new Error(`Unexpected job status: ${statusResponse.status}`);
        
      } catch (error) {
        console.error(`Error polling job ${jobId}:`, error.message);
        // Continue polling for transient errors, but with longer interval
        currentInterval = Math.min(currentInterval * 2, maxInterval);
      }
    }
    
    throw new Error(`Job ${jobId} timed out after ${timeout/1000} seconds`);
  }
}

Training Operations

Handle training workflows with proper state monitoring:

class TrainingManager {
  async triggerAndMonitorTraining(diagramId, nodeId, options = {}) {
    const {
      checkInterval = 60000,    // 1 minute for training checks
      timeout = 3600000,        // 1 hour timeout
      maxStateChanges = 10      // Prevent infinite loops
    } = options;

    // Trigger training
    await this.triggerTraining(diagramId, nodeId);
    console.log(`Training initiated for diagram ${diagramId}`);
    
    let lastState = null;
    let stateChangeCount = 0;
    const startTime = Date.now();
    
    while (Date.now() - startTime < timeout) {
      const agentStatus = await this.getAgentStatus(diagramId);
      const currentState = agentStatus.stateId;
      
      // Track state changes to detect issues
      if (currentState !== lastState) {
        stateChangeCount++;
        if (stateChangeCount > maxStateChanges) {
          throw new Error('Training failed: Too many state changes detected');
        }
        console.log(`Training state changed: ${lastState} → ${currentState}`);
        lastState = currentState;
      }
      
      switch (currentState) {
        case 6: // Training in progress
          console.log('Agent is training...');
          await new Promise(resolve => setTimeout(resolve, checkInterval));
          break;
          
        case 2: // Idle/Ready
          console.log('Training completed successfully!');
          return { success: true, finalState: currentState };
          
        default:
          // Unexpected state - log but continue monitoring briefly
          console.warn(`Unexpected training state: ${currentState}`);
          await new Promise(resolve => setTimeout(resolve, checkInterval));
      }
    }
    
    throw new Error(`Training monitoring timed out after ${timeout/1000} seconds`);
  }
}

Performance Optimization

Connection Pooling and Reuse

Reuse HTTP connections when making multiple API calls to reduce latency and improve performance.

// Using node-fetch with keep-alive
const fetch = require('node-fetch');
const Agent = require('agentkeepalive');

const agent = new Agent({
  maxSockets: 100,
  maxFreeSockets: 10,
  timeout: 60000,
  freeSocketTimeout: 30000
});

const apiClient = {
  async makeRequest(url, options) {
    return fetch(url, {
      ...options,
      agent: agent
    });
  }
};

Intelligent Caching Strategies

Cache responses appropriately based on operation type:

class IntelligentCache {
  constructor() {
    this.cache = new Map();
    // Different TTL for different operation types
    this.ttlMap = {
      'agent_status': 30000,      // 30 seconds - changes during training
      'diagram_info': 300000,     // 5 minutes - relatively static
      'job_status': 5000,         // 5 seconds - changes frequently
      'auth_token': 86400000      // 24 hours - matches token expiry
    };
  }

  set(key, value, operationType = 'default') {
    const ttl = this.ttlMap[operationType] || 60000; // 1 minute default
    
    this.cache.set(key, {
      value,
      timestamp: Date.now(),
      ttl
    });
  }

  get(key) {
    const item = this.cache.get(key);
    if (!item) return null;
    
    if (Date.now() - item.timestamp > item.ttl) {
      this.cache.delete(key);
      return null;
    }
    
    return item.value;
  }

  // Clear cache for specific diagram when training starts
  invalidateTrainingCache(diagramId) {
    const keysToDelete = Array.from(this.cache.keys())
      .filter(key => key.includes(`diagram_${diagramId}`));
    
    keysToDelete.forEach(key => this.cache.delete(key));
    console.log(`Invalidated ${keysToDelete.length} cache entries for diagram ${diagramId}`);
  }
}

Rate Limiting Compliance

Respect API rate limits by implementing proper throttling and backoff strategies.

class RateLimiter {
  constructor(requestsPerSecond = 10) {
    this.requestsPerSecond = requestsPerSecond;
    this.requests = [];
  }

  async throttle() {
    const now = Date.now();
    
    // Remove requests older than 1 second
    this.requests = this.requests.filter(time => now - time < 1000);
    
    if (this.requests.length >= this.requestsPerSecond) {
      const oldestRequest = Math.min(...this.requests);
      const waitTime = 1000 - (now - oldestRequest);
      await new Promise(resolve => setTimeout(resolve, waitTime));
    }
    
    this.requests.push(now);
  }
}

Error Handling and Resilience

Comprehensive Error Handling

Handle both HTTP errors and cognitive diagram execution errors:

class APIError extends Error {
  constructor(message, statusCode, errorType, context = {}) {
    super(message);
    this.name = 'APIError';
    this.statusCode = statusCode;
    this.errorType = errorType;
    this.context = context;
    this.timestamp = new Date().toISOString();
  }
}

async function handleAPIResponse(response, context = {}) {
  if (!response.ok) {
    const errorBody = await response.json().catch(() => ({}));
    throw new APIError(
      errorBody.message || response.statusText,
      response.status,
      'HTTP_ERROR',
      context
    );
  }

  const data = await response.json();
  
  // Critical: errorData may contain info messages, not just errors
  if (data.errorData && data.errorData !== "[]") {
    try {
      const errorList = JSON.parse(data.errorData);
      const actualErrors = errorList.filter(item => 
        item.type === 'ERROR' || (!item.type && item.message.toLowerCase().includes('error'))
      );
      
      if (actualErrors.length > 0) {
        throw new APIError(
          actualErrors.map(e => `${e.node_name || 'Unknown'}: ${e.message}`).join('; '),
          200,
          'DIAGRAM_ERROR',
          { ...context, errorData: errorList }
        );
      }
      
      // Log info messages but don't treat as errors
      const infoMessages = errorList.filter(item => item.type === 'INFO');
      if (infoMessages.length > 0) {
        console.log('Diagram info messages:', infoMessages.map(m => m.message));
      }
      
    } catch (parseError) {
      console.warn('Failed to parse errorData:', data.errorData);
    }
  }

  return data;
}

Circuit Breaker Pattern

Implement circuit breaker pattern to prevent cascading failures when the API is experiencing issues.

class CircuitBreaker {
  constructor(threshold = 5, timeout = 60000) {
    this.threshold = threshold;
    this.timeout = timeout;
    this.failureCount = 0;
    this.lastFailureTime = null;
    this.state = 'CLOSED'; // CLOSED, OPEN, HALF_OPEN
  }

  async execute(operation) {
    if (this.state === 'OPEN') {
      if (Date.now() - this.lastFailureTime > this.timeout) {
        this.state = 'HALF_OPEN';
      } else {
        throw new Error('Circuit breaker is OPEN');
      }
    }

    try {
      const result = await operation();
      this.onSuccess();
      return result;
    } catch (error) {
      this.onFailure();
      throw error;
    }
  }

  onSuccess() {
    this.failureCount = 0;
    this.state = 'CLOSED';
  }

  onFailure() {
    this.failureCount++;
    this.lastFailureTime = Date.now();
    
    if (this.failureCount >= this.threshold) {
      this.state = 'OPEN';
    }
  }
}

Concurrency and Resource Management

Concurrency Control

Manage concurrent operations to avoid overwhelming the API or your application:

class ConcurrencyManager {
  constructor(maxConcurrent = 5) {
    this.maxConcurrent = maxConcurrent;
    this.running = 0;
    this.queue = [];
  }

  async execute(operation) {
    return new Promise((resolve, reject) => {
      this.queue.push({ operation, resolve, reject });
      this.processQueue();
    });
  }

  async processQueue() {
    if (this.running >= this.maxConcurrent || this.queue.length === 0) {
      return;
    }

    this.running++;
    const { operation, resolve, reject } = this.queue.shift();

    try {
      const result = await operation();
      resolve(result);
    } catch (error) {
      reject(error);
    } finally {
      this.running--;
      this.processQueue(); // Process next item in queue
    }
  }
}

// Usage for multiple diagram executions
const concurrencyManager = new ConcurrencyManager(3); // Max 3 concurrent requests

const results = await Promise.all(
  messages.map(message => 
    concurrencyManager.execute(() => 
      executeCognitiveDiagram(token, diagramId, message)
    )
  )
);

Resource Cleanup and Memory Management

Implement proper cleanup for long-running applications:

class ResourceManager {
  constructor() {
    this.timers = new Set();
    this.intervals = new Set();
    this.connections = new Set();
  }

  setTimeout(callback, delay) {
    const timer = setTimeout(() => {
      this.timers.delete(timer);
      callback();
    }, delay);
    this.timers.add(timer);
    return timer;
  }

  setInterval(callback, interval) {
    const timer = setInterval(callback, interval);
    this.intervals.add(timer);
    return timer;
  }

  addConnection(connection) {
    this.connections.add(connection);
  }

  cleanup() {
    // Clear all timers
    this.timers.forEach(timer => clearTimeout(timer));
    this.intervals.forEach(timer => clearInterval(timer));
    
    // Close all connections
    this.connections.forEach(conn => {
      if (conn.destroy) conn.destroy();
      if (conn.close) conn.close();
    });

    // Clear collections
    this.timers.clear();
    this.intervals.clear();
    this.connections.clear();
    
    console.log('Resources cleaned up successfully');
  }
}

// Global cleanup handler
const resourceManager = new ResourceManager();

process.on('SIGINT', () => {
  console.log('Received SIGINT, cleaning up...');
  resourceManager.cleanup();
  process.exit(0);
});

API Usage Patterns

Batch Processing

Process multiple requests efficiently by implementing proper batching and concurrency control.

async function processBatch(items, processor, concurrency = 5) {
  const results = [];
  
  for (let i = 0; i < items.length; i += concurrency) {
    const batch = items.slice(i, i + concurrency);
    const batchPromises = batch.map(processor);
    const batchResults = await Promise.allSettled(batchPromises);
    results.push(...batchResults);
  }
  
  return results;
}

// Usage example
const messages = ['Hello', 'How are you?', 'Goodbye'];
const results = await processBatch(
  messages,
  message => executeCognitiveDiagram(token, diagramId, message),
  3 // Process 3 at a time
);

Conversation Management

Maintain conversation context effectively by using conversation IDs and proper state management.

class ConversationManager {
  constructor() {
    this.conversations = new Map();
  }

  createConversation(userId) {
    const conversationId = `conv_${userId}_${Date.now()}`;
    this.conversations.set(conversationId, {
      userId,
      messages: [],
      createdAt: new Date(),
      lastActivity: new Date()
    });
    return conversationId;
  }

  addMessage(conversationId, message, response) {
    const conversation = this.conversations.get(conversationId);
    if (conversation) {
      conversation.messages.push({ message, response, timestamp: new Date() });
      conversation.lastActivity = new Date();
    }
  }

  getContext(conversationId, maxMessages = 5) {
    const conversation = this.conversations.get(conversationId);
    if (!conversation) return null;
    
    return conversation.messages
      .slice(-maxMessages)
      .map(m => `User: ${m.message}\nAI: ${m.response}`)
      .join('\n\n');
  }
}

Monitoring and Logging

Comprehensive Logging Strategy

Implement structured logging for debugging and monitoring:

const logger = {
  info: (message, metadata = {}) => {
    console.log(JSON.stringify({
      level: 'INFO',
      message,
      timestamp: new Date().toISOString(),
      ...metadata
    }));
  },
  
  error: (message, error, metadata = {}) => {
    console.error(JSON.stringify({
      level: 'ERROR',
      message,
      error: error.message,
      stack: error.stack,
      timestamp: new Date().toISOString(),
      ...metadata
    }));
  },

  // Special logging for job operations
  jobStatus: (jobId, status, metadata = {}) => {
    console.log(JSON.stringify({
      level: 'INFO',
      message: 'Job status update',
      jobId,
      status,
      timestamp: new Date().toISOString(),
      ...metadata
    }));
  },

  // Training-specific logging
  training: (diagramId, state, metadata = {}) => {
    console.log(JSON.stringify({
      level: 'INFO',
      message: 'Training status update',
      diagramId,
      state,
      timestamp: new Date().toISOString(),
      ...metadata
    }));
  }
};

Performance Monitoring

Track API performance metrics to identify bottlenecks and optimize your integration.

class PerformanceMonitor {
  constructor() {
    this.metrics = {
      requestCount: 0,
      totalResponseTime: 0,
      errorCount: 0,
      operationMetrics: new Map() // Track by operation type
    };
  }

  async trackRequest(operation, operationType = 'unknown') {
    const startTime = Date.now();
    this.metrics.requestCount++;
    
    // Initialize operation metrics if not exists
    if (!this.metrics.operationMetrics.has(operationType)) {
      this.metrics.operationMetrics.set(operationType, {
        count: 0,
        totalTime: 0,
        errors: 0
      });
    }
    
    const opMetrics = this.metrics.operationMetrics.get(operationType);
    opMetrics.count++;
    
    try {
      const result = await operation();
      const responseTime = Date.now() - startTime;
      
      this.metrics.totalResponseTime += responseTime;
      opMetrics.totalTime += responseTime;
      
      logger.info('API request completed', {
        operationType,
        responseTime,
        avgResponseTime: this.metrics.totalResponseTime / this.metrics.requestCount,
        opAvgTime: opMetrics.totalTime / opMetrics.count
      });
      
      return result;
    } catch (error) {
      this.metrics.errorCount++;
      opMetrics.errors++;
      
      logger.error('API request failed', error, {
        operationType,
        responseTime: Date.now() - startTime,
        errorRate: this.metrics.errorCount / this.metrics.requestCount,
        opErrorRate: opMetrics.errors / opMetrics.count
      });
      throw error;
    }
  }

  getMetricsSummary() {
    const summary = {
      overall: {
        totalRequests: this.metrics.requestCount,
        totalErrors: this.metrics.errorCount,
        avgResponseTime: this.metrics.totalResponseTime / this.metrics.requestCount,
        errorRate: this.metrics.errorCount / this.metrics.requestCount
      },
      byOperation: {}
    };

    this.metrics.operationMetrics.forEach((metrics, operation) => {
      summary.byOperation[operation] = {
        count: metrics.count,
        avgTime: metrics.totalTime / metrics.count,
        errorRate: metrics.errors / metrics.count
      };
    });

    return summary;
  }
}

Testing Strategies

Unit Testing

Write comprehensive tests for your API integration logic.

// Example using Jest
describe('Synthreo API Client', () => {
  let client;
  
  beforeEach(() => {
    client = new SynthreoAPIClient(mockAuthClient);
  });

  test('should handle authentication errors gracefully', async () => {
    mockAuthClient.getAccessToken.mockRejectedValue(new Error('Invalid credentials'));
    
    await expect(client.executeDiagram('123', {}))
      .rejects
      .toThrow('Invalid credentials');
  });

  test('should parse successful responses correctly', async () => {
    const mockResponse = {
      result: 'OK',
      outputData: '[{"response": "Hello World"}]',
      errorData: '[]'
    };
    
    fetch.mockResolvedValue({
      ok: true,
      json: () => Promise.resolve(mockResponse)
    });

    const result = await client.executeDiagram('123', {});
    expect(result.outputData).toBe('[{"response": "Hello World"}]');
  });

  test('should handle job polling correctly', async () => {
    const jobId = 'test-job-123';
    
    // Mock job still running, then completed
    fetch
      .mockResolvedValueOnce({
        status: 202,
        json: () => Promise.resolve({ job: { id: jobId, status: 1 }})
      })
      .mockResolvedValueOnce({
        status: 200,
        json: () => Promise.resolve({ result: 'OK', outputData: 'Job completed' })
      });

    const result = await client.pollJobStatus(jobId, 100); // Fast polling for test
    expect(result.outputData).toBe('Job completed');
  });
});

Environment-Specific Configurations

Development vs Production Settings

Configure your application differently for different environments:

const config = {
  development: {
    polling: {
      jobInterval: 5000,        // 5 seconds for faster development
      trainingInterval: 30000,  // 30 seconds
      timeout: 300000          // 5 minutes
    },
    logging: {
      level: 'debug',
      verbose: true
    },
    retry: {
      maxAttempts: 3,
      baseDelay: 1000
    }
  },
  
  production: {
    polling: {
      jobInterval: 30000,       // 30 seconds for production efficiency
      trainingInterval: 60000,  // 1 minute
      timeout: 3600000         // 1 hour
    },
    logging: {
      level: 'info',
      verbose: false
    },
    retry: {
      maxAttempts: 5,
      baseDelay: 2000
    }
  }
};

const env = process.env.NODE_ENV || 'development';
const currentConfig = config[env];

Security Checklist

Pre-Production Security Review

Before deploying your Synthreo API integration to production:

• Credentials are stored in environment variables or secure vaults
• No API keys, passwords, or tokens in source code
• All API communications use HTTPS
• Token refresh logic is implemented and tested
• Input validation and sanitization is in place
• Error messages don't expose sensitive information
• Logging excludes sensitive data (tokens, passwords)
• Rate limiting and timeout configurations are appropriate
• Circuit breaker patterns are implemented for resilience
• Resource cleanup is properly handled

By following these comprehensive best practices, you'll build more reliable, secure, and maintainable integrations with the Synthreo Builder API that can handle the complexities of real-world production environments.