Connect

Work with Jira Issues

The Jira processor enables querying Jira issues using JQL (Jira Query Language) and returning structured data. Unlike traditional connectors, Jira is implemented as a processor only, which can be used as both an input (with generate) and an output (with drop).

Prerequisites

Redpanda CLI

Install the Redpanda CLI (rpk) to run Redpanda Connect. See Get Started with Redpanda Connect using rpk for installation instructions.

Enterprise license

The Jira processor requires a Redpanda Enterprise Edition license. See Enterprise Licensing for instructions on generating a trial license and setting the REDPANDA_LICENSE environment variable.

Jira environment variables

The examples in this cookbook use environment variables for Jira credentials and configuration. This allows you to keep sensitive credentials secure and separate from your pipeline configuration files.

Generate a Jira API token.

Set up your environment variables:

JIRA_BASE_URL=https://<domain>.atlassian.net (1)
JIRA_USERNAME=<email-address> (2)
JIRA_API_TOKEN=<api-token> (3)
JIRA_PROJECT=<project-key> (4)

1	Your Jira instance URL. You can find this in your browser when logged into Jira.
2	Your Jira account email address.
3	The API token generated from your Atlassian account.
4	The key of the Jira project you want to query (such as `DOC` or `K8S`).

You can export these variables in your shell or place them in a .env file for use with rpk connect run --env-file <env-file>.

Use Jira as an input

To use Jira as an input, combine the generate input with the Jira processor. This pattern triggers Jira queries at regular intervals or on-demand.

Query Jira periodically

This example queries Jira every 30 seconds for recent issues:

input:
  generate:
    mapping: |
      root.jql = "project = ${JIRA_PROJECT} AND created >= -1d ORDER BY created DESC"
      root.maxResults = 50
      root.fields = ["key", "summary", "status", "assignee", "created"]
    interval: 30s

pipeline:
  processors:
    - jira:
        base_url: "${JIRA_BASE_URL}"
        username: "${JIRA_USERNAME}"
        api_token: "${JIRA_API_TOKEN}"
        max_results_per_page: 50
        timeout: 30s
        backoff:
          max_retries: 10

    # Transform each issue
    - mapping: |
        root.issue_key = this.key
        root.summary = this.fields.summary
        root.status = this.fields.status.name
        root.assignee = if this.fields.assignee != null {
          this.fields.assignee.displayName
        } else { "Unassigned" }
        root.created = this.fields.created

output:
  stdout:
    codec: lines

One-time query

For a single query, use count instead of interval:

input:
  generate:
    mapping: |
      root.jql = "project = ${JIRA_PROJECT} ORDER BY created DESC"
      root.maxResults = 100
      root.fields = ["key", "summary", "status"]
    count: 1  # Run only once

pipeline:
  processors:
    - jira:
        base_url: "${JIRA_BASE_URL}"
        username: "${JIRA_USERNAME}"
        api_token: "${JIRA_API_TOKEN}"

output:
  stdout: {}

Advanced query with custom fields

input:
  generate:
    mapping: |
      root.jql = "assignee = currentUser() AND status IN (Open, 'In Progress') ORDER BY updated DESC"
      root.maxResults = 20
      root.fields = [
        "key",
        "summary",
        "status",
        "assignee",
        "priority",
        "created",
        "updated",
        "customfield_10001"  # Custom field ID
      ]
      root.startAt = 0
    interval: 60s

pipeline:
  processors:
    - jira:
        base_url: "${JIRA_BASE_URL}"
        username: "${JIRA_USERNAME}"
        api_token: "${JIRA_API_TOKEN}"
        max_results_per_page: 100

    # Transform each individual issue message
    - mapping: |
        root.key = this.key
        root.summary = this.fields.summary
        root.status = this.fields.status.name
        root.priority = this.fields.priority.name
        root.custom_field = this.fields.customfield_10001

output:
  kafka:
    addresses: ["localhost:9092"]
    topic: jira-issues

Use Jira as an output

To use Jira as an output, place the Jira processor before a drop output. This executes queries without forwarding results downstream.

Query and drop results

input:
  stdin:
    codec: lines

pipeline:
  processors:
    # Prepare Jira query from incoming message
    - mapping: |
        root.jql = this.jql.or("project = ${JIRA_PROJECT} AND status = Open")
        root.maxResults = this.maxResults.or(100)
        root.fields = this.fields.or(["key", "summary", "status"])

    # Execute Jira query
    - jira:
        base_url: "${JIRA_BASE_URL}"
        username: "${JIRA_USERNAME}"
        api_token: "${JIRA_API_TOKEN}"
        backoff:
          max_retries: 3
        timeout: 30s

    # Log each issue before dropping
    - log:
        message: "Processing Jira issue: ${! this.key } - ${! this.fields.summary }"
        level: INFO

output:
  drop: {}

Trigger queries from Kafka

input:
  kafka:
    addresses: ["localhost:9092"]
    topics: ["jira-query-requests"]
    consumer_group: jira-query-consumer

pipeline:
  processors:
    # Parse incoming query request
    - mapping: |
        root.jql = this.query
        root.maxResults = 50

    # Execute query
    - jira:
        base_url: "${JIRA_BASE_URL}"
        username: "${JIRA_USERNAME}"
        api_token: "${JIRA_API_TOKEN}"

    # Process each individual issue
    - mapping: |
        root.query_id = uuid_v4()
        root.timestamp = now()
        root.issue_key = this.key
        root.issue_summary = this.fields.summary

output:
  drop: {}

Input message format

The Jira processor expects input messages containing valid Jira queries in JSON format:

{
  "jql": "project = MYPROJECT AND status = Open",
  "maxResults": 50,
  "fields": ["key", "summary", "status", "assignee"]
}

Required fields

jql: The JQL (Jira Query Language) query string

Optional fields

maxResults: Maximum number of results to return (default: 50)
fields: Array of field names to include in the response

Configuration options

For a complete list of configuration options, see Jira Processor Reference.

JQL query language

The Jira processor supports querying Jira issues using JQL (Jira Query Language). JQL is a flexible, text-based query language similar to SQL that allows you to search for issues matching specific criteria.

Common JQL patterns

Here are common JQL patterns for filtering issues:

Recent issues by project

project = <YOUR_PROJECT> AND created >= -7d ORDER BY created DESC

Issues assigned to current user

assignee = currentUser() AND status != Done

Issues by status

project = <YOUR_PROJECT> AND status IN (Open, 'In Progress', 'To Do')

Issues by priority

project = <YOUR_PROJECT> AND priority = High ORDER BY created DESC

Issues updated recently

project = <YOUR_PROJECT> AND updated >= -1d ORDER BY updated DESC

Run the examples

Each example in this cookbook can be saved as a YAML file and run individually. The examples use environment variables for credentials and configuration, so you don’t need to modify the YAML files.

Set up your environment:

# Set Jira credentials
export JIRA_BASE_URL=https://your-domain.atlassian.net
export JIRA_USERNAME=your-email@example.com
export JIRA_API_TOKEN=your-api-token
export JIRA_PROJECT=YOUR_PROJECT

Save any example to a YAML file (for example, jira-query.yaml).
Run the example:
```
rpk connect run jira-query.yaml
```

Output message format

The Jira processor returns individual issue messages, rather than a response object with an issues array.

Each message output by the Jira processor represents a single issue:

{
  "id": "12345",
  "key": "DOC-123",
  "fields": {
    "summary": "Example issue",
    "status": {
      "name": "In Progress"
    },
    "assignee": {
      "displayName": "John Doe"
    },
    "created": "2025-01-15T10:30:00.000-0800",
    "updated": "2025-01-16T14:20:00.000-0800"
  }
}

Process output messages

When processing messages from the Jira processor, keep in mind:

Do not use unarchive to split an issues array
Do not reference this.issues or this.total in your mappings
Do access fields directly such as this.key, this.fields.summary.

Handle null fields

Some Jira fields can be null and require conditional handling:

pipeline:
  processors:
    - mapping: |
        # Assignee can be null
        root.assignee = if this.fields.assignee != null {
          this.fields.assignee.displayName
        } else { "Unassigned" }

        # Reporter can be null
        root.reporter = if this.fields.reporter != null {
          this.fields.reporter.displayName
        } else { "Unknown" }

        # Resolution date is null for open issues
        root.resolved_at = this.fields.resolutiondate

        # Calculate lead time only for resolved issues
        root.lead_time_days = if this.fields.resolutiondate != null {
          (this.fields.resolutiondate.ts_parse("2006-01-02T15:04:05.999-0700").ts_unix() -
           this.fields.created.ts_parse("2006-01-02T15:04:05.999-0700").ts_unix()) / 86400
        } else { null }

Pagination handling

The Jira processor automatically handles pagination internally using the max_results_per_page setting. The processor:

Makes the initial request with startAt=0.
Checks if more results are available.
Automatically fetches subsequent pages until all results are retrieved.
Outputs each issue as an individual message.

You don’t need to handle pagination manually. The processor streams all matching issues as separate messages.

Error handling

The Jira processor includes built-in retry logic for rate limiting (HTTP 429) and configurable timeouts. See Jira Processor Reference for detailed configuration options.

Create and update Jira issues

The Jira processor is read-only and only supports querying. To create or update Jira issues, you must use the http processor with the Jira REST API.

Create a Jira issue

Use the HTTP processor to POST to the Jira REST API /issue endpoint. This example creates a Task issue with a title and description:

# Create Jira Issue with HTTP Processor
#
# The Jira processor is read-only and cannot create or update issues.
# Use the HTTP processor to create Jira issues using the REST API.

input:
  generate:
    mapping: |
      root = {
        "title": "Example issue from Redpanda Connect",
        "description": "This issue was created using the HTTP processor",
        "issue_type": "Task"
      }
    count: 1

pipeline:
  processors:
    - mapping: |
        root = {
          "fields": {
            "project": {
              "key": "${JIRA_PROJECT}"
            },
            "summary": this.title,
            "description": {
              "type": "doc",
              "version": 1,
              "content": [
                {
                  "type": "paragraph",
                  "content": [
                    {
                      "type": "text",
                      "text": this.description
                    }
                  ]
                }
              ]
            },
            "issuetype": {
              "name": this.issue_type
            }
          }
        }

    - label: create_issue
      http:
        url: "${JIRA_BASE_URL}/rest/api/3/issue"
        verb: POST
        headers:
          Content-Type: application/json
          Accept: application/json
        basic_auth:
          enabled: true
          username: "${JIRA_USERNAME}"
          password: "${JIRA_API_TOKEN}"

    - mapping: |
        root.issue_key = this.key
        root.issue_id = this.id
        root.self_link = this.self

output:
  stdout:
    codec: lines

tests:
  - name: Transform data for creating issue
    target_processors: '/pipeline/processors/0'
    environment:
      JIRA_PROJECT: "TEST"
    input_batch:
      - json_content:
          title: "Test issue"
          description: "Test description"
          issue_type: "Bug"
    output_batches:
      - - json_equals:
            fields:
              project:
                key: "TEST"
              summary: "Test issue"
              description:
                type: "doc"
                version: 1
                content:
                  - type: "paragraph"
                    content:
                      - type: "text"
                        text: "Test description"
              issuetype:
                name: "Bug"

  - name: Mock create issue API response
    target_processors: '/pipeline/processors'
    mocks:
      create_issue:
        mapping: |
          root.id = "10001"
          root.key = "DOC-789"
          root.self = "https://your-domain.atlassian.net/rest/api/3/issue/10001"
    input_batch:
      - json_content:
          title: "Test issue"
          description: "Test description"
          issue_type: "Task"
    output_batches:
      - - json_equals:
            issue_key: "DOC-789"
            issue_id: "10001"
            self_link: "https://your-domain.atlassian.net/rest/api/3/issue/10001"

Atlassian Document Format (ADF): The description field uses ADF, Jira’s rich text format. For simple text, wrap it in a paragraph structure as shown.
Issue type: Common types are Task, Bug, Story, Epic. The type must exist in your Jira project.
Required fields: At minimum, you need project, summary, and issuetype. Other fields may be required based on your Jira configuration.
Response: The API returns the created issue’s key, id, and self (URL).

Troubleshoot

License errors

If you receive license-related errors:

Verify the license is exported:
```
echo $REDPANDA_LICENSE
```

Regenerate if needed:

rpk generate license --name "<first-name>" \
  --last-name "<last-name>" \
  --email "<email>" \
  --company "<company>"
export REDPANDA_LICENSE=$(cat ./redpanda.license)

Check license expiration:

The trial license expires after 30 days. Check the output from rpk generate license for the expiration date.

Authentication errors

If you receive 401 Unauthorized errors:

Verify your API token is correct
Ensure you’re using your email address as the username
Check that your Jira instance URL is correct (include https://)

Rate limiting

If you hit Jira rate limits (HTTP 429):

Increase max_retries in the Jira processor config
Reduce query frequency by increasing interval
Use more specific JQL queries to reduce result sizes

Query performance

For large result sets:

Use more specific JQL queries
Limit fields with the fields parameter
Reduce max_results_per_page if memory is constrained

Suggested labs

Stream Jira Issues to Redpanda for Real-Time Metrics

Search all labs

Was this helpful?

group Ask in the community

mail Share your feedback

group_add Make a contribution