Connect a system

Connect an external system through an ESB route using secure credential references and supported protocols.

Wire an external system into ESB as a route that the mediation engine can invoke. A route binds a service + operation to an outbound endpoint over a supported protocol. This guide covers choosing a protocol, referencing credentials (never pasting plaintext), testing connectivity, and deploying.

Prerequisites

  • A running ESB (see Getting Started).
  • A target system reachable from the ESB container.
  • For protocol UI setup: the Connectors view in the admin console.
  • A stored secret reference for any credential the endpoint needs (the Secret References page in the admin console, backed by Vault, environment variables or AWS as configured in enterprise.secrets.provider).

Step 1 — Choose a protocol

ESB supports the connector protocols listed below. Pick by what the remote system speaks:

ProtocolUse whenUI surface
RESTHTTP(S) JSON/any API, path-param expansionRoutes API / config.yaml
GRPCgRPC services (proto reflection or descriptor set)Routes API / config.yaml
SOAPXML/SOAP 1.1 web services with fault detectionRoutes API / config.yaml
SQLRead-only or parameterized SQL against a DBRoutes API / config.yaml
KAFKAProduce records to a Kafka topicRoutes API / config.yaml
AMQPPublish to RabbitMQ exchanges/queuesRoutes API / config.yaml
MQTTPublish to an MQTT 3.1.1 brokerRoutes API / config.yaml
QUEUEEnqueue to the internal ESB queueInternal broker
EVENTPublish to the internal event busInternal broker
SFTPDeliver payloads as files (atomic temp+rename)Connector Wizard
SMTPSend payloads as emailConnector Wizard

The admin Connector Wizard currently covers SFTP and SMTP; every other protocol is configured through the routes surface (admin API or config.yaml).

Step 2 — Store credentials as secret references

Never embed passwords, tokens, or private keys in a route. Store them once and reference by name:

Open Secret References in the admin console, choose Add Secret Reference, and give it a name (for example smtp-primary), a provider and the external name the provider knows the secret by.

The secret value is resolved at runtime from the configured provider; the route holds only the reference. On the Credentials step of the Connector Wizard you pick a stored secret reference the same way.

Step 3 — Configure the route (Connector Wizard)

The wizard walks Endpoint → Credentials → Delivery → Security → Review.

Endpoint: name the route (lowercase, a-z0-9_-), pick the protocol (SFTP — deliver files, or SMTP — send email), the service/operation it maps to, and the provider id (used for health checks and circuit breakers), plus the endpoint URL with scheme and port, e.g. sftp://files.example.com:2022 or smtp://smtp.example.com:587.

Credentials: choose the secret reference. The wizard stores endpoint.auth.secret_ref and never persists the value.

Delivery: for SFTP the base directory on the remote host; for SMTP the from address and recipient list.

Security: SFTP host-key verification (known_hosts file or a pinned fingerprint — insecure_host_key is only for non-production), and SMTP TLS (implicit TLS, STARTTLS, or require_tls).

The wizard’s Test step validates the configuration shape; there is no live connectivity probe yet, so connectivity is verified on first deploy.

Step 4 — Configure other protocols via config.yaml

For REST, gRPC, SOAP, SQL, Kafka, AMQP, and MQTT, declare the route in config.yaml. Example — a REST route with an API-key secret reference, health check, and circuit breaker:

routes:
  - id: route-orders-rest
    endpoint:
      provider_id: orders-rest
      url: "http://inventory.internal:8080/api/orders"
      method: POST
      protocol: REST
      timeout: 30s
      auth:
        type: api-key-header
        header: X-API-Key
        secret_ref:
          ref: inventory-api-key
      health_check:
        url: "http://inventory.internal:8080/health"
        interval: 30s
      circuit_breaker:
        enabled: true
        failure_threshold: 5
        success_threshold: 2
        open_duration: 30s
    conditions:
      - source: payload
        path: $.service
        equals: orders

Then validate and deploy the config:

./esbctl config validate config.yaml
./esbctl login --username admin   # once; stores a session
./esbctl config deploy config.yaml

Step 5 — Test connectivity and deploy

  • Route test: ./esbctl route test /api/orders --service orders --operation create --payload test-order.json validates route selection only: it resolves the service/operation against the route conditions and reports the matched route id, candidate route ids, and rejected routes (with reasons). No provider or connector is invoked — nothing is sent over the wire.
  • Health: each connector reports a Health state; the reliability layer surfaces per-provider health in the dashboard and drives health-aware route selection and circuit breakers.
  • Deploy: the config deploy above activates the route. Successful connector calls are recorded as mediation metrics (protocol, operation, outcome).

Success signals and recovery

StageSuccess signalFailure recovery
Secret reference storedsecret_references lists itValue must resolve from the provider; check enterprise.secrets.provider.
Route validatedesbctl config validate passesMissing secret_ref on a credential-bearing protocol → error at validate time.
Route deployedRoute appears in GET /api/v1/admin/routesConfig deploy failed → inspect the error and re-validate.
Connector call2xx Response; mediation metric outcome: okTransient transport failures are retryable, permanent config errors are not.
HealthProvider healthy in dashboardCheck endpoint reachability, TLS material, and secret resolution.

Troubleshooting: if a call fails, open the Operations → correlation timeline for the request’s correlation id (GET /api/v1/admin/correlation/{id}) to see the ingress, route, policy, and connector hops and their outcomes in order.