Write Integration Tests for Transform Functions

Learn how to write integration tests for data transform functions in Redpanda, including setting up unit tests and using testcontainers for integration tests.

This guide covers how to write both unit tests and integration tests for your transform functions. While unit tests focus on testing individual components in isolation, integration tests verify that the components work together as expected in a real environment.

Unit tests

You can create unit tests for transform functions by mocking the interfaces injected into the transform function and asserting that the input and output work correctly. This typically includes mocking the WriteEvent and RecordWriter interfaces.

package main

import (
  "testing"
  "github.com/stretchr/testify/assert"
  "github.com/stretchr/testify/mock"
  "github.com/redpanda-data/redpanda/src/transform-sdk/go/transform"
)

// MockWriteEvent is a mock implementation of the WriteEvent interface.
type MockWriteEvent struct {
  mock.Mock
}

func (m *MockWriteEvent) Record() transform.Record {
  args := m.Called()
  return args.Get(0).(transform.Record)
}

// MockRecordWriter is a mock implementation of the RecordWriter interface.
type MockRecordWriter struct {
  mock.Mock
}

func (m *MockRecordWriter) Write(record transform.Record) error {
  args := m.Called(record)
  return args.Error(0)
}

// copyRecord copies the record to the output topic.
func copyRecord(event transform.WriteEvent, writer transform.RecordWriter) error {
  record := event.Record()
  return writer.Write(record)
}

// TestCopyRecord tests the copyRecord function.
func TestCopyRecord(t *testing.T) {
  // Create mocks for the WriteEvent and RecordWriter
  event := new(MockWriteEvent)
  writer := new(MockRecordWriter)

  // Set up the expected behavior
  record := transform.Record{Value: []byte("test")}
  event.On("Record").Return(record)
  writer.On("Write", record).Return(nil)

  // Call the function under test
  err := copyRecord(event, writer)

  // Assert that no error occurred and that the expectations were met
  assert.NoError(t, err)
  event.AssertExpectations(t)
  writer.AssertExpectations(t)
}

To run your unit tests, use the following command:

go test

This will execute all tests in the current directory.

Integration tests

Integration tests verify that your transform functions work correctly in a real Redpanda environment. You can use testcontainers to set up and manage a Redpanda instance for testing.

For more detailed examples and helper code for setting up integration tests, refer to the SDK integration tests on GitHub.