Maximo integration, from a sensor reading to a work order: MIF, JMS, Kafka and MQTT explained

Part 1The map: two kinds of integration

Before any product name, separate two jobs. Almost every integration question in a Maximo project is one of them.

Job 1 is the territory of the Maximo Integration Framework inside Manage, with JMS queues (or Kafka) behind it. Job 2 is the territory of MQTT, the IoT tool and Monitor, with Kafka in the middle. The two meet when Monitor decides that a trend deserves attention and creates a service request or a work order in Manage.

Part 2Four words that get mixed up

Part 3Kafka on OpenShift

On OpenShift nobody installs Kafka by hand. An operator does it. You write a short description of the cluster you want (a custom resource of kind Kafka), and the operator creates the pods, the storage, the certificates and the network routes, and keeps them that way.

The open-source operator for Kafka is called Strimzi. Red Hat's supported product, Streams for Apache Kafka (until recently named AMQ Streams), is built from Strimzi, and the resources look the same: Kafka, KafkaTopic, KafkaUser. The system used for this guide runs the Strimzi operator 0.45 with Kafka 3.9, installed by the suite's own installer. Everything below applies to both.

That single resource becomes nine pods: three brokers, three ZooKeeper nodes, the operator itself, an entity operator (it watches topic and user resources) and an exporter for metrics.

The five Kafka words

Three settings of this cluster are worth knowing, because they explain behaviour you will see later:

• Messages are kept for 24 hours (or until a partition reaches 1 GiB). Kafka here is a pipe with a memory of one day, not an archive.
• Topics are not created automatically. A program that writes to a topic that does not exist gets an error. Somebody must create the topic first.
• Clients must log in with a user and password (SCRAM-SHA-512) over TLS. Inside the cluster the address is maskafka-kafka-bootstrap.strimzi.svc:9093; from outside it is an OpenShift route on port 443.

Exercise 1: create a topic, send five messages, read them twice

You need: a login to the OpenShift cluster with the oc command, Python 3 on your laptop, and the Kafka user name, password and CA certificate. On a suite cluster they are in the secret that the suite's Kafka configuration points to, and in the status of the Kafka resource.

Step 1. Create the topic the OpenShift way. A topic is a resource too. Save this as topic.yaml:

YAMLapiVersion: kafka.strimzi.io/v1beta2
kind: KafkaTopic
metadata:
  name: demo.pump.alerts
  namespace: strimzi
  labels:
    strimzi.io/cluster: maskafka      # which Kafka cluster owns the topic
spec:
  partitions: 3
  replicas: 3
  config:
    retention.ms: 86400000            # keep messages for one day

Shell$ oc apply -f topic.yaml
kafkatopic.kafka.strimzi.io/demo.pump.alerts created

$ oc get kafkatopic -n strimzi
NAME               CLUSTER    PARTITIONS   REPLICATION FACTOR   READY
demo.pump.alerts   maskafka   3            3                    True

Step 2. Send five messages. Install a client with pip install kafka-python-ng. The connection settings are the same for every program, so they go in one small file, kc.py:

PythonBOOT = "maskafka-kafka-tls-bootstrap-strimzi.apps.<your-cluster-domain>:443"

def cfg():
    return dict(bootstrap_servers=BOOT, security_protocol="SASL_SSL", sasl_mechanism="SCRAM-SHA-512",
                sasl_plain_username=open("user.txt").read().strip(),
                sasl_plain_password=open("password.txt").read().strip(),
                ssl_cafile="kafka-ca.crt")

The producer gives every message a key, the pump's name. Kafka uses the key to choose the partition:

Python# produce.py
import json
from kafka import KafkaProducer
from kc import cfg

p = KafkaProducer(**cfg(), key_serializer=str.encode,
                  value_serializer=lambda v: json.dumps(v).encode(), acks="all")
for i, pump in enumerate(["PUMP-001", "PUMP-002", "PUMP-001", "PUMP-003", "PUMP-001"]):
    md = p.send("demo.pump.alerts", key=pump,
                value={"pump": pump, "alert": "HIGH_VIBRATION", "seq": i}).get(15)
    print(f"sent key={pump} seq={i} -> partition {md.partition} offset {md.offset}")
p.flush()

Shell$ python produce.py
sent key=PUMP-001 seq=0 -> partition 1 offset 0
sent key=PUMP-002 seq=1 -> partition 2 offset 0
sent key=PUMP-001 seq=2 -> partition 1 offset 1
sent key=PUMP-003 seq=3 -> partition 0 offset 0
sent key=PUMP-001 seq=4 -> partition 1 offset 2

Step 3. Read them, as a named group.

Python# consume.py
import sys
from kafka import KafkaConsumer
from kc import cfg

group = sys.argv[1]
c = KafkaConsumer("demo.pump.alerts", **cfg(), group_id=group,
                  auto_offset_reset="earliest", consumer_timeout_ms=12000)
n = 0
for m in c:
    n += 1
    print(f"[{group}] partition {m.partition} offset {m.offset} key={m.key.decode()} value={m.value.decode()}")
print(f"[{group}] read {n} messages")
c.close()

Shell$ python consume.py demo-readers
[demo-readers] partition 1 offset 0 key=PUMP-001 value={"pump": "PUMP-001", "alert": "HIGH_VIBRATION", "seq": 0}
[demo-readers] partition 1 offset 1 key=PUMP-001 value={"pump": "PUMP-001", "alert": "HIGH_VIBRATION", "seq": 2}
[demo-readers] partition 1 offset 2 key=PUMP-001 value={"pump": "PUMP-001", "alert": "HIGH_VIBRATION", "seq": 4}
[demo-readers] partition 2 offset 0 key=PUMP-002 value={"pump": "PUMP-002", "alert": "HIGH_VIBRATION", "seq": 1}
[demo-readers] partition 0 offset 0 key=PUMP-003 value={"pump": "PUMP-003", "alert": "HIGH_VIBRATION", "seq": 3}
[demo-readers] read 5 messages

$ python consume.py demo-readers        # the same group again
[demo-readers] read 0 messages

$ python consume.py audit-team          # a different group
...
[audit-team] read 5 messages

Where is a Kafka message stored?

On the disk of the brokers, as plain files. Each broker has a persistent volume. Each partition is a directory named <topic>-<partition>, and inside it the messages are appended to a segment file ending in .log. You can look at it from the OpenShift console: open a broker pod, choose the Terminal tab, and list the directory. The tool kafka-dump-log.sh, which ships with Kafka, prints what a segment file contains.

So "where are the messages processed and stored?" has a precise answer for Kafka: stored in segment files on the brokers' volumes for the retention time, copied to three brokers; processed by whichever consumer groups read the topic, each at its own pace.

Part 4MQTT and the IoT tool, without the mystery

MQTT is the protocol small devices use to send data. It was designed for weak hardware and poor networks, which is why it is everywhere in industry. Its whole vocabulary is five words:

What the IoT tool really is

In the suite catalogue it is one tile called IoT. On OpenShift it is a handful of pods, and once you know their names the mystery is gone:

And here is the part that links this chapter to the previous one. The IoT tool does not store your readings. It receives them over MQTT and writes them to a Kafka topic. Monitor is a Kafka consumer that reads that topic and puts the readings in its database.

Exercise 2: register a pump, publish over MQTT, find the reading in Kafka

Since version 9 of the suite, devices are registered in Monitor, under Setup. There are two concepts:

• A device type is the template: "a cooling pump reports these four numbers".
• A device is one real thing of that type, with its own identity and secret token.

Step 1. Create the device type. In Monitor open Setup > Device types > Add device type. Choose Empty template. The sample types on the same page are worth a look later: they come with simulated data.

Name it CoolingPump. The name cannot be changed later, so decide your naming rule before the first one.

Step 2. Add the metrics. On the Metrics tab choose Add metric, once per number the pump reports. Every metric belongs to an event, which is simply the name of the message the device sends. Here one event called status carries all four numbers.

Step 3. Register a device. Open Setup > Devices > Add device. The first dialog is where gateways come in (Part 5); for a device that connects on its own, choose Add custom device and leave the gateway empty.

Give it the ID PUMP-001, leave the token on Auto-generated, and pick the type CoolingPump. When you save, Monitor shows the credentials once.

Step 4. Publish. A device proves who it is with three values. They follow a fixed pattern:

With pip install paho-mqtt, a pump is twenty lines of Python:

Python# pump.py
import json, random, ssl, time
import paho.mqtt.client as mqtt

HOST = "wsp.messaging.iot.<your-suite-domain>"
c = mqtt.Client(mqtt.CallbackAPIVersion.VERSION2, client_id="d:wsp:CoolingPump:PUMP-001")
c.username_pw_set("use-token-auth", open("PUMP-001.token").read().strip())
ctx = ssl.create_default_context()
ctx.check_hostname = False; ctx.verify_mode = ssl.CERT_NONE   # lab only: load the suite CA instead
c.tls_set_context(ctx)
c.on_connect = lambda cl, u, flags, rc, props=None: print("connected:", rc)
c.connect(HOST, 443, keepalive=60)
c.loop_start(); time.sleep(2)

while True:
    reading = {"temperature": round(62 + random.uniform(-1, 1), 2),
               "pressure":    round(4.1 + random.uniform(-.1, .1), 3),
               "vibration":   round(2.1 + random.uniform(-.25, .25), 3),
               "flow":        round(120 + random.uniform(-2, 2), 2)}
    c.publish("iot-2/evt/status/fmt/json", json.dumps(reading), qos=1).wait_for_publish(10)
    print(time.strftime("%H:%M:%S"), reading)
    time.sleep(10)

Shell$ python pump.py
connected: Success
07:51:37 {'temperature': 62.47, 'pressure': 4.022, 'vibration': 2.299, 'flow': 121.07}
07:51:40 {'temperature': 62.37, 'pressure': 4.051, 'vibration': 1.995, 'flow': 121.13}
07:51:42 {'temperature': 62.87, 'pressure': 4.072, 'vibration': 2.069, 'flow': 121.49}

Step 5. See it arrive. In Monitor open the device and its Recent event tab.

Step 6. Find the same reading in Kafka. This is the step that makes the architecture real. Read the end of the topic instdb2_topic_events with a few lines of Python (same kc.py as in exercise 1):

Shell$ python peek.py instdb2_topic_events 2
partition 0: offsets 0..0
partition 1: offsets 0..3
partition 2: offsets 0..0
p1 o1 key=b'…\x00\x03wsp\x00\x0bCoolingPump\x00\x08PUMP-001\x00\x06status\x00\x04json'
   value=b'{"temperature": 62.37, "pressure": 4.051, "vibration": 1.995, "flow": 121.13}'
p1 o2 key=b'…\x00\x03wsp\x00\x0bCoolingPump\x00\x08PUMP-001\x00\x06status\x00\x04json'
   value=b'{"temperature": 62.87, "pressure": 4.072, "vibration": 2.069, "flow": 121.49}'

MQTT or Kafka? The question behind most confusion

Both "send messages to topics", so they sound like rivals. They are not. They solve opposite halves of the same journey, and almost every industrial architecture uses both, in this order: MQTT from the device to the first server, Kafka from that server to everything else.

You have already seen every row of that table in this guide. The pump script connected with a device identity and a token and published one small JSON document: MQTT. The same document then sat in a file on a broker's disk at offset 17, read by Monitor's consumer group and readable again by yours: Kafka.

Part 5Gateways: how plant data reaches the suite

A pump does not speak MQTT, and it certainly does not hold a token. In a real plant something stands between the equipment and the suite. That something is called a gateway, and the word covers several quite different things. The question to ask is always the same: who holds the connection, and who is known to the suite?

In Monitor a gateway is registered under Setup > Gateways > Add gateway. Like a device it gets an ID, a type and a token. The form then asks one question that devices do not have: its Configuration.

The device library: when someone has already done the modelling

In exercise 2 everything about the pump was typed by hand: the device type, the event, four metrics and their units. That is fine for one pump you invented. It is slow and error-prone for a real drive or power meter that exposes dozens of values in numbered registers. For those, Monitor ships a device library: a catalogue of real, manufacturer-specific products, each with its protocol settings and its metrics already described.

The way in is the same Add device button. The dialog offers two tiles, Use device library and Add custom device. The first one has a condition printed under it: "Only managed gateways are available. Library devices must be added to a managed gateway." On this system the list was empty until a gateway with the Managed configuration existed, so one was created first (ID EDGE-GW-01, type EdgeGateway). After that, Continue opens a wizard of five steps.

Two details on that last screen are worth understanding. Data frequency is how often the gateway reads the value. Change threshold is how much it must change before it is worth sending; "On every change" sends each new value. Together they decide how much traffic one pump creates, which matters when there are five hundred of them on one site network.

The fourth way in has its own page too: Setup > Data ingestion is a list of uploaded files, with the device type each one fed, its status and its record count.

Exercise 5: Node-RED, the pump redrawn as a flow

Exercise 2 used forty lines of Python to be a pump. Plenty of people who need to connect equipment do not write Python. Node-RED is the tool they reach for. It is open source, it started at IBM and now lives at the OpenJS Foundation, and it runs anywhere from a laptop to a small industrial gateway. You work in a browser: drag nodes from a palette onto a canvas, join them with wires, and press Deploy. A message travels along the wires from left to right, and each node does one thing to it.

Get it running

One container is enough. This is the command, with podman on a Windows laptop; docker takes the same arguments. (Here a folder on the laptop was mounted at /data in place of the named volume, which makes no difference to the exercise.) The version that started was Node-RED 5.0.8.

Shell$ podman run -d --name nodered -p 1880:1880 -v nodered-data:/data docker.io/nodered/node-red:latest
$ podman logs nodered | grep -E "version|Started"
13 Oct 10:46:10 - [info] Node-RED version: v5.0.8
13 Oct 10:46:10 - [info] Node.js  version: v24.21.0
13 Oct 10:46:11 - [info] Started flows

The editor is then at http://localhost:1880.

Flow 1: the pump, as five nodes

The function node holds the only code in the exercise:

JavaScript// One pump reading. While flow.fault is true the bearing wears a little more on every reading.
const fault = flow.get('fault') || false;
let wear = fault ? (flow.get('wear') || 0) + 0.25 : 0;
flow.set('wear', wear);
const t = Date.now() / 1000;
const jitter = (n) => (Math.random() - 0.5) * n;
msg.payload = {
    temperature: +(61 + 3 * Math.sin(t / 100) + jitter(1.2) + wear * 2.5).toFixed(2),
    pressure:    +(4.2 + 0.15 * Math.sin(t / 70) + jitter(0.1) - wear * 0.08).toFixed(3),
    vibration:   +(2.1 + jitter(0.5) + wear).toFixed(3),
    flow:        +(118 + 4 * Math.sin(t / 125) + jitter(2) - wear * 3).toFixed(2)
};
msg.topic = 'iot-2/evt/status/fmt/json';
node.status({ fill: fault ? 'red' : 'green', shape: 'dot', text: 'vibration ' + msg.payload.vibration });
return msg;

Everything that was a line of connection code in Python is a field in the mqtt out node's server settings. The values are the same five from exercise 2, because the device has not changed:

Press Deploy. Under the mqtt node a green dot and the word connected appear, and the debug panel starts to fill. To break the bearing, click the small square on the left of Start bearing fault.

Monitor cannot tell who is on the other end. It sees a device called PUMP-001 sending status events, exactly as before:

Flow 2: a small gateway

Part 5 said a gateway accepts data in whatever form the plant has and passes it on as device events. Here is one in four nodes. Imagine an older system that can only send a line of text over HTTP: temperature,pressure,vibration,flow.

JavaScript// A legacy system posts one line of text: temperature,pressure,vibration,flow
const p = String(msg.payload).trim().split(',').map(Number);
if (p.length !== 4 || p.some(isNaN)) {
    msg.statusCode = 400;
    msg.payload = { error: 'expected: temperature,pressure,vibration,flow' };
    return [null, msg];
}
const reading = { temperature: p[0], pressure: p[1], vibration: p[2], flow: p[3] };
const toIot = { topic: 'iot-2/evt/status/fmt/json', payload: reading };
msg.statusCode = 202;
msg.payload = { accepted: true, device: 'PUMP-001', reading: reading };
return [toIot, msg];

Three calls show the three behaviours: no login, a good line, a bad line.

Shell$ curl -s -o /dev/null -w "%{http_code}\n" -H "Content-Type: text/plain" -d "70.5,3.9,2.4,111" http://localhost:1880/pump/reading
401
$ curl -s -u pump:<password> -H "Content-Type: text/plain" -d "70.5,3.9,2.4,111" http://localhost:1880/pump/reading
{"accepted":true,"device":"PUMP-001","reading":{"temperature":70.5,"pressure":3.9,"vibration":2.4,"flow":111}}
$ curl -s -u pump:<password> -H "Content-Type: text/plain" -d "hello" http://localhost:1880/pump/reading
{"error":"expected: temperature,pressure,vibration,flow"}

That is a gateway in miniature: a protocol the suite does not speak on one side, MQTT on the other, and a place in the middle to check and reshape the data. A bad line is refused at the door with a 400 and never reaches Monitor.

Where Node-RED fits, and where it does not

Part 6Monitor: from readings to decisions

So far the readings only travel. Monitor is where they start to mean something. It does four things with them, in this order:

Step 1 of 3: look at the data

Open the device and its Data tab, then pick a metric, for example Bearing vibration. Monitor draws the trend for the chosen period: last 24 hours, last month, and so on. For a short period it does not draw every reading. It draws two lines per metric, the highest and the lowest value of each hour, which is the right view for spotting drift.

In this exercise the Python script played the part of a wearing bearing. After 25 minutes of normal operation it raised the vibration a little with every reading, from about 2 mm/s to more than 8, and let temperature rise and flow fall with it. Then the "repair" happened and the values went back to normal. On the trend, the maximum line climbs away from the minimum line. Nobody told Monitor about any of that. The question is whether the system can notice by itself.

Step 2 of 3: teach Monitor what "too high" means

On the device type, the tab Calculated metrics holds everything Monitor works out by itself. Add calculated metric opens a catalogue of ready-made functions. A calculation defined on the device type applies to every device of that type: one rule for a thousand pumps.

For a first alert, AlertHighValue is enough: "fire alert when metric exceeds an upper threshold". It asks three things on three screens:

Step 3 of 3: from alert to work

Every device type gets a dashboard called Alerts without you building it. A few minutes after the rule was saved, the indicator at the top of the device type changed from "Analysis off" to "Analysis on", and the dashboard was no longer empty. Open it from Monitor data on the Monitor home page, then the device type.

Three things on this screen are worth a minute each.

• It found the past. The rule was created after the bearing episode was over. The first analysis run went back over the stored readings and raised the alerts with their original times, from about 08:22 to 08:37. Data that is stored can be judged again with a better rule; data that was never kept cannot.
• Ninety alerts for one problem. The simple function fires for every reading above the limit, and the pump reported every ten seconds for a quarter of an hour. One failing bearing, ninety rows. This is the "alert noise" that makes operators stop looking. In a real setup you would alert on an hourly maximum, or use a function that fires when a condition has occurred several times within a window, such as AlertByOccurrencesCount in the same catalogue.
• The last column is the bridge to Manage. From the row, an operator sets an owner, changes the status from New to Acknowledged, Validated or Resolved, and with Create service request opens a record in Manage for the maintenance team.

That closes the loop this guide started with: a number left a device over MQTT, crossed Kafka, was stored and judged by Monitor, and can arrive in Manage as a piece of work for a person.

Bonus step: build your own dashboard

The Alerts dashboard is given to you. Everything else you compose yourself from cards, with no code. On the device type, next to the Alerts tab, click the + tab, type a title (here Pump health) and click Configure dashboard. The editor opens with an empty canvas on the left and a Gallery on the right.

Click a card in the gallery and it lands on the canvas. Then, in the right panel, give it a title, a size and a Data item: the list offers the four metrics defined earlier (Temperature, Discharge pressure, Bearing vibration, Flow rate). While you edit, the cards are drawn with made-up sample values; the banner at the top says so. Click Save and close and the same cards fill with the real readings.

The dashboard below has nine cards: four Value/KPI cards for "now", four Time series line cards for the trend, and one Alert table.

Now read it the way an operator would. The numbers in the top row are all normal, so nothing is wrong now. The charts tell what happened earlier:

• Vibration climbs from 2 to almost 9 mm/s shortly after 08:20 and drops back at once: the bearing episode from the exercise. A smaller second episode follows just after 10:00.
• Temperature rises to 80 °C in the same minutes, and pressure falls from 4.2 to 3.5 bar.
• The Open alerts table lists the vibration_high rows that the rule raised for those minutes.

Three charts moving together in the same minutes is what turns "a high number" into a diagnosis. One reading above a limit can be a bad sensor. Vibration up, temperature up and pressure down at the same moment is a pump in trouble. That is the reason to put related metrics next to each other on one page.

JSON{
  "title": "Temperature now",
  "size": "SMALL",
  "type": "VALUE",
  "content": {
    "attributes": [ { "dataSourceId": "temperature_1", "label": "Temperature", "unit": "°C", "precision": 2 } ]
  },
  "dataSource": {
    "attributes": [ { "id": "temperature_1", "attribute": "temperature", "aggregator": "last", "eventName": "status" } ]
  }
}

Read it from the bottom: take the attribute temperature from the event status, keep the last value in the selected time range, and show it with two decimals and the unit. Change last to max or mean and the same card answers a different question.

Going further: let Monitor find what "not normal" looks like

The alert in step 2 needed a number from you: 3.5 mm/s. Somebody has to know that number, for every metric of every kind of equipment. Anomaly detection turns the question around. You do not say what is too high. A function looks at the recent behaviour of the signal and gives every reading a score for how unlike the rest it is. You then alert on the score.

These functions sit in the same catalogue as AlertHighValue. The catalogue is larger than it looks in the dialog; IBM lists and describes every entry on one page, Built-in functions. The families worth recognising:

Add an anomaly score to the pump

On the device type, open Setup, the tab Calculated metrics, and click Add calculated metric. Type "Anomaly" in the filter.

Choose KMeansAnomalyScore and Select. The wizard has the same three steps as the alert:

1. Scope: all devices of this type.
2. Input: input_item is the metric to watch, here vibration. windowsize is how many consecutive readings are judged together; the dialog itself says it "is typically set to 12 data points".
3. Output: a name for the new data item, here vibration_anomaly, with Auto schedule on.

The new item appears in the list of calculated metrics with category Transformer: it creates data, not alerts. Because it is a data item like any other, it can go on a dashboard. In the dashboard editor a Time series line card offers vibration_anomaly in its Data item list, next to the four raw metrics.

The first result was a flat line, and why

The card showed a perfectly flat score of zero, even while a new bearing episode was being sent. A flat zero is not "everything is normal". It was the function failing quietly. Monitor runs its calculations as a short-lived pod every five minutes in the namespace mas-instdb2-monitor, and that pod's log says what happened:

Shell$ oc logs -n mas-instdb2-monitor <latest wsp.g1... pod> | grep -i kmeans
KMeans, Entity: PUMP-001, Input: vibration, Windowsize: 12, Output: ['vibration_anomaly'], Overlap: 6, Inputsize: 30
KMeans failed with Could not form valid cluster separation. Please change n_clusters or change clustering method

Read the end of the first line: Inputsize: 30. Each run hands the function only the readings that arrived since the previous run. The pump was reporting every ten seconds, so a five-minute run contained about thirty values. With a window of 12 that is a handful of windows, too few to tell normal from unusual, and the function gives up and writes zero.

Now read the two charts together.

• The score has no unit. It is not mm/s. It only means "more unusual" when higher. Here normal running sits between 0 and about 0.5, and the fault reaches about 1.4, roughly three times the baseline.
• It reacts to shape, not to level. Nobody told the function about 3.5 mm/s. It reacted because a steady climb looks nothing like the usual jitter around 2.
• It is noisy. The baseline never rests, and during the episode there are three separate spikes with dips between them. One reading of the score above a limit is weak evidence. Several within a few minutes is strong evidence, which is exactly what AlertByOccurrencesCount is for.
• The first point of the baseline, near 1.0 at 12:22, is a start-up effect of the first run that had enough data. Expect that after any gap, and do not alert on the first minutes.

Which detector, and what about supervised?

The dialog's own advice is honest: "Try several anomaly models on your data and use the one that fits." IBM's page Unsupervised anomaly detectors explains each one. A short guide:

All of these are unsupervised: they need no examples of failure, only data. The other kind is supervised. IBM documents one, GBMRegressor, on the page Supervised anomaly detectors. You name a target metric and the features that should explain it, for example predict discharge pressure from flow and temperature. The function trains a model on real data and then predicts the target. It has no alert of its own; you add one that fires when prediction and reality differ by more than a threshold. Use it when you understand which metrics drive which, and retrain it when the equipment or its firmware changes.

Why a business pays for this

Juniors are often asked to "connect the sensors" without being told why. These are the reasons a plant manager cares, in the order they usually appear:

Part 7The Maximo Integration Framework

Now the other lane: business records. Everything here lives in Manage, under the Integration module. The framework has seven building blocks. Learn them as a chain, because that is how a message travels.

Sequential or continuous: the question every interviewer asks

There is also a path with no queue at all: the REST API on object structures (/maximo/api/os/…). The caller waits, gets the answer immediately, and handles errors itself. New integrations often start there. Queues earn their place when the caller must not wait, when the partner may be offline, or when a failed message must be kept and replayed.

Part 8Switching on JMS in a containerised Manage

This is the part that surprises people who learned Maximo 7.6. There, an administrator created the queues in WebSphere. In Maximo Application Suite, Manage runs as containers that an operator builds and replaces, so a change made inside a running server is gone at the next restart. The queues must be declared to the operator.

And out of the box, they are not there. Open Integration > External Systems and the action Add/Modify Queues: the list shows nine queues, which looks complete. But these are only records in the Maximo database, each holding a JNDI name. On the system used here, nothing stood behind those names. The server configuration had the JMS client feature only, and its one JMS-related file was a commented-out sample:

Shell$ oc exec -n mas-instdb2-manage <manage-pod> -- grep -n -i jms /config/server.xml
10:      <feature>wasJmsClient-2.0</feature>
16:      <!--feature>wmqJmsClient-2.0</feature>
17:      <feature>jmsMdb-3.2</feature-->

Where the setting lives: the Manage workspace resource

Manage is described by a custom resource of kind ManageWorkspace, in the project mas-<instance>-manage. The suite's administration pages edit this same resource: Administration > Workspace > Manage > Actions > Update configuration > Server bundles. Whichever way you go in, four pieces under spec.settings.deployment make a working JMS setup. None of them is enough alone.

Step 1. The client configuration. Save this as server-custom.xml. Only one value is specific to your system: the address of the JMS server, which is the name of its OpenShift service, <instance>-<workspace>-<bundle name>.mas-<instance>-manage.svc, on port 7276.

XML<?xml version="1.0" encoding="UTF-8"?>
<server description="Manage all bundle: client of the JMS server bundle">
  <featureManager>
    <feature>jndi-1.0</feature>
    <feature>wasJmsClient-2.0</feature>
    <feature>jmsMdb-3.2</feature>
    <feature>mdb-3.2</feature>
  </featureManager>

  <!-- one connection factory for all MIF queues, pointing at the JMS server pod through its service -->
  <jmsQueueConnectionFactory jndiName="jms/maximo/int/cf/intcf" connectionManagerRef="mifjmsconfact">
    <properties.wasJms remoteServerAddress="instdb2-wsp-jms.mas-instdb2-manage.svc:7276:BootstrapBasicMessaging"/>
  </jmsQueueConnectionFactory>
  <connectionManager id="mifjmsconfact" maxPoolSize="20"/>

  <!-- JNDI name used by the queue records in Manage  ->  queue defined on the JMS server -->
  <jmsQueue id="jms/maximo/int/queues/sqout" jndiName="jms/maximo/int/queues/sqout">
    <properties.wasJms queueName="sqoutbd"/>
  </jmsQueue>
  <jmsQueue id="jms/maximo/int/queues/sqin" jndiName="jms/maximo/int/queues/sqin">
    <properties.wasJms queueName="sqinbd"/>
  </jmsQueue>
  <jmsQueue id="jms/maximo/int/queues/cqin" jndiName="jms/maximo/int/queues/cqin">
    <properties.wasJms queueName="cqinbd"/>
  </jmsQueue>
  <jmsQueue id="jms/maximo/int/queues/cqinerr" jndiName="jms/maximo/int/queues/cqinerr">
    <properties.wasJms queueName="cqinerrbd"/>
  </jmsQueue>
  <jmsQueue id="jms/maximo/int/queues/cqout" jndiName="jms/maximo/int/queues/cqout">
    <properties.wasJms queueName="cqoutbd"/>
  </jmsQueue>
  <jmsQueue id="jms/maximo/int/queues/cqouterr" jndiName="jms/maximo/int/queues/cqouterr">
    <properties.wasJms queueName="cqouterrbd"/>
  </jmsQueue>
  <jmsQueue id="jms/maximo/int/queues/notf" jndiName="jms/maximo/int/queues/notf">
    <properties.wasJms queueName="notfbd"/>
  </jmsQueue>
  <jmsQueue id="jms/maximo/int/queues/notferr" jndiName="jms/maximo/int/queues/notferr">
    <properties.wasJms queueName="notferrbd"/>
  </jmsQueue>

  <!-- consumers of the continuous queues: message-driven beans inside the Manage server -->
  <jmsActivationSpec id="maximo-all/mboejb/JMSContQueueProcessor-1" maxEndpoints="5">
    <properties.wasJms destinationRef="jms/maximo/int/queues/cqin" destinationType="javax.jms.Queue"
                       remoteServerAddress="instdb2-wsp-jms.mas-instdb2-manage.svc:7276:BootstrapBasicMessaging"/>
  </jmsActivationSpec>
  <jmsActivationSpec id="maximo-all/mboejb/JMSContQueueProcessor-2" maxEndpoints="1">
    <properties.wasJms destinationRef="jms/maximo/int/queues/cqinerr" destinationType="javax.jms.Queue"
                       remoteServerAddress="instdb2-wsp-jms.mas-instdb2-manage.svc:7276:BootstrapBasicMessaging"/>
  </jmsActivationSpec>
  <!-- the same two again for cqout and cqouterr: JMSContOutQueueProcessor-1 and -2 -->
</server>

Notice what this file is: the bridge between the two halves you have already seen. On the left, the JNDI names that Manage's queue records hold (jms/maximo/int/queues/sqout). On the right, the queues that exist on the JMS server (sqoutbd). The file is stored in OpenShift as a secret:

Shell$ oc create secret generic wsp-all-jms-client -n mas-instdb2-manage \
     --from-file=server-custom.xml=server-custom.xml
secret/wsp-all-jms-client created

Step 2. The workspace. This is the change to the ManageWorkspace resource. It can be applied as a patch, typed into the YAML editor of the OpenShift console, or entered on the suite's Update configuration page:

YAML$ oc patch manageworkspace instdb2-wsp -n mas-instdb2-manage --type merge -p '
spec:
  settings:
    deployment:
      defaultJMS: true                         # message beans for the continuous queues
      persistentVolumes:
        - pvcName: manage-jms
          mountPath: /jmsstore                 # the path the JMS server writes its store to
          size: 5Gi
          storageClassName: ocs-external-storagecluster-cephfs
          accessModes: [ReadWriteMany]
      serverBundles:
        - name: all                            # the existing Manage server bundle ...
          bundleType: all
          isDefault: true
          isMobileTarget: true
          isUserSyncTarget: true
          replica: 1
          routeSubDomain: all
          additionalServerConfig:
            secretName: wsp-all-jms-client     # ... now with the client XML of step 1
        - name: jms                            # new: the JMS server
          bundleType: standalonejms
          isDefault: false
          isMobileTarget: false
          isUserSyncTarget: false
          replica: 1
          routeSubDomain: jms
'

What happens after you save, and how to read the status page

The change is not instant. Adding a bundle makes the operator rebuild the Manage images and roll the pods. On this cluster the image build took about fifteen minutes, and each roll-out of the pods a few more. The suite shows the progress under Administration > Workspace > Manage.

This page confuses everybody the first time, because it seems to say two opposite things: "The application is deployed and in a ready state" with a green tick, and "Activating in workspace" with a spinner. Both are true. They describe two different resources:

Each tile is one status condition of the ManageWorkspace resource. You can read the same six lines with oc get manageworkspace … -o yaml, under status.conditions.

The time under each tile is when that condition last changed, which is the quickest way to see what is moving. When all tiles are green and the spinner is gone, the page switches from "Activation in progress" to the configuration summary.

What you get: the JMS server and its queues

That terminal output is worth reading line by line, because it is the physical answer to "where is my queue?":

This is where Maximo's queued messages physically are: in PermanentStore, on that volume, until a consumer takes them. The generated configuration of the pod is short enough to read in full, and it answers most questions about Maximo queues:

XML<featureManager>
    <feature>wasJmsSecurity-1.0</feature>
    <feature>wasJmsServer-1.0</feature>                       <!-- this pod IS the messaging server -->
</featureManager>
<wasJmsEndpoint host="*" wasJmsSSLPort="7286" wasJmsPort="7276" />   <!-- Manage connects here -->
<messagingEngine>
    <fileStore path="/jmsstore"/>                             <!-- where messages are kept -->
    <queue id="sqoutbd"   maintainStrictOrder="true" maxMessageDepth="500000"
           failedDeliveryPolicy="KEEP_TRYING" maxRedeliveryCount="-1"/>
    <queue id="sqinbd"    maintainStrictOrder="true" maxMessageDepth="500000"
           failedDeliveryPolicy="KEEP_TRYING" maxRedeliveryCount="-1"/>
    <queue id="cqinerrbd" maxMessageDepth="500000" failedDeliveryPolicy="KEEP_TRYING"/>
    <queue id="cqinbd"    maxMessageDepth="500000" exceptionDestination="cqinerrbd"/>
    <queue id="cqouterrbd" maxMessageDepth="500000" failedDeliveryPolicy="KEEP_TRYING"/>
    <queue id="cqoutbd"   maxMessageDepth="500000" exceptionDestination="cqouterrbd"/>
    <queue id="notferrbd" maxMessageDepth="100000" failedDeliveryPolicy="KEEP_TRYING"/>
    <queue id="notfbd"    maxMessageDepth="100000" exceptionDestination="notferrbd"/>
</messagingEngine>

Read it against the table of Part 7 and the theory becomes configuration:

• The two sequential queues have maintainStrictOrder="true" and KEEP_TRYING with unlimited redelivery: a failing message is retried in place, and nothing overtakes it.
• The continuous queues have an exceptionDestination: a failing message is moved to the error queue and the rest carry on.
• maxMessageDepth is the size of the waiting room: 500,000 messages per queue.
• The names end in bd (bus destination). Manage's queue records point to them through JNDI names such as jms/maximo/int/queues/sqout.

Part 9Exercises 3 and 4: a message out, a message in

With a messaging server in place, the framework can be used the way the diagrams of Part 7 show. Both exercises use the same partner, a pretend ERP.

Set up the partner

Step 1. The external system. In Integration > External Systems choose New External System. Name it DEMOERP, pick an end point, pick the queues, tick Enabled, save. The end point here is the delivered MXXMLFILE, which simply writes each outbound message to a file. It is the easiest one to verify, and swapping it later for an HTTP or Kafka end point changes nothing else.

Step 2. What flows. On the tab Publish Channels add MXASSETInterface and tick Enabled: assets go out. On the tab Enterprise Services add MXASSETInterface, tick Enabled, and clear Use Continuous Queue? so that inbound assets use the sequential queue and arrive in order. Save.

Step 3. Switch the channel on. A publish channel only reacts to saves when its listener is enabled. Open Integration > Publish Channels, find MXASSETInterface, and choose Enable Event Listener. While you are there, choose Message Tracking, tick Enable Message Tracking and Store Message, and save.

Exercise 3: a message out

Step 1. Create an asset in Assets: number PUMP-001, description "Cooling water pump 1". Save. For the user, that is all. Nothing on the screen says that a message was produced.

Step 2. Find the message in the queue. Back in External Systems, open Add/Modify Queues, select the row jms/maximo/int/queues/sqout, and choose Actions > View Queue Data with Count Only? ticked.

One message, and it is not moving. Why? Because nothing is reading the queue yet.

Step 3. Start the consumer. Sequential queues are read by a cron task, and on a new system its instances are not active. Open System Configuration > Platform Configuration > Cron Task Setup, find JMSQSEQCONSUMER, tick Active? for SEQQOUT and SEQQIN, and save.

Within a minute the server log shows the hand-over, and the end point has written its file:

Output[CID-CRON-437] Message 2530637.179187229887528509 processed by JMS Queue Cron Task
               JMSQSEQCONSUMER.SEQQOUT, destjndiname = jms/maximo/int/queues/sqout

$ oc exec -n mas-instdb2-manage <manage-pod> -- ls -la /opt/ibm/wlp/output/defaultServer/xmlfiles/
-rw-r-----. 1 1000960000 root 7888 Oct 13 06:29 DEMOERP_MXASSETInterface_2530637.179187229887528509.xml

The file is the asset as the partner would receive it. Shortened:

XML<PublishMXASSET xmlns="http://www.ibm.com/maximo" creationDateTime="2026-10-13T06:18:18+00:00"
                messageID="1453117918722980361594" event="1">
  <MXASSETSet>
    <ASSET action="Add">
      <ASSETNUM>PUMP-001</ASSETNUM>
      <DESCRIPTION>Cooling water pump 1</DESCRIPTION>
      <SITEID>BEDFORD</SITEID>
      <STATUS>NOT READY</STATUS>
      ... about 200 more fields ...
    </ASSET>
  </MXASSETSet>
</PublishMXASSET>

Step 4. The history. The queue is empty again, so where is the proof? In Integration > Message Tracking, because tracking was switched on for the channel.

Exercise 4: a message in

Now the ERP sends an asset. An enterprise service can be reached in several ways: over HTTP at /meaweb/es/<external system>/<service>, from a file, from an interface table, from a queue. On the suite the HTTP door answers 401 API Key required to an unidentified caller, so a real partner is given an API key. For a first test there is a door that needs no key: the Data Import action of the external system. It puts the file's content through the same queue and the same processing.

Step 1. Save this as pump-002.xml:

XML<?xml version="1.0" encoding="UTF-8"?>
<SyncMXASSET xmlns="http://www.ibm.com/maximo">
  <MXASSETSet>
    <ASSET>
      <ASSETNUM>PUMP-002</ASSETNUM>
      <DESCRIPTION>Cooling water pump 2 (sent by the ERP)</DESCRIPTION>
      <SITEID>BEDFORD</SITEID>
    </ASSET>
  </MXASSETSet>
</SyncMXASSET>

Three fields are enough. The object structure fills in the rest with the same defaults and validations as the Assets application. Sync means "create it if it is new, update it if it exists".

Step 2. In External Systems open DEMOERP, tab Enterprise Services, select the row, and choose Actions > Data Import. Pick the file, leave XML File selected, and confirm.

Manage answers "Inbound transaction has been processed successfully to queue." Read that sentence carefully. It says to queue. The asset does not exist yet.

Step 3. It fails. And it did, on this system. No asset appeared. The place to look is Integration > Message Reprocessing.

The server log tells the same story, with the full text:

Output[CID-CRON-697] got message 9635157.1791873245883959498 from queue jms/maximo/int/queues/sqin
[ERROR] Processing failed for MXASSETInterface. ...
        BMXAA9301E - The user of the transaction is not authorized for Object Structure MXASSET.
        Configure authorization in the object structure application and grant necessary access to the user.
[ERROR] Message 9635157.1791873245883959498 processing was unsucessful by JMS Queue Cron Task
        JMSQSEQCONSUMER.SEQQIN

Step 4. Fix the cause, not the message. An object structure has to be told whose security it follows. In Integration > Object Structures open MXASSET, choose Configure Object Structure Security, set Authorization Name to the application ASSET with the lookup, confirm and save. From now on the object structure follows the security of the Assets application: a user who may create assets on the screen may create them through the integration.

Step 5. Replay. Back in Message Reprocessing, tick the message and choose Process Selected Messages. The list empties, and the asset is there.

And Kafka, for the queues of Manage?

Manage can also use Kafka instead of JMS for its integration queues. The action Add/Modify Message Providers in External Systems is where a Kafka cluster is declared (servers, user, password, security settings), and each queue record has a Message Provider column that says which provider it lives on. The framework above it stays the same: the same channels, services and Message Reprocessing. It is worth considering when Kafka is already the company's backbone, because the partner systems can then read Maximo's topics directly. I did not run that configuration for this guide, so take this paragraph as a pointer, not as a tested recipe.

Part 10Where every message lives

The question a support engineer asks first is "where is my message right now?". Here is the whole guide on one page.

Part 11When it does not work

ValueWho gains what

Queues, topics and brokers are plumbing. This is what the plumbing buys.

No figures are promised here. Record your own baseline for each measure before the change, and compare after one quarter.

GlossaryThe words, once more

Broker

A messaging server. Kafka has brokers; MQTT has a broker; the JMS server is one too.

Consumer group

A named set of Kafka readers sharing one position per partition.

Custom resource

A description of something you want on OpenShift (a Kafka cluster, a topic, a Manage workspace) that an operator turns into running pods.

Device type / device

The template and the individual thing in Monitor.

End point

The destination of an outbound MIF message and the handler that delivers it.

Enterprise service

The inbound side of MIF for one object structure.

External system

A partner in MIF: its channels, services and queues.

Gateway

Anything that connects to the suite on behalf of devices that cannot.

Object structure

The definition of a business document in Maximo; the base of MIF and of the REST API.

Offset

The position of a message in a Kafka partition.

Operator

Software on OpenShift that installs and looks after another piece of software.

Partition

One ordered slice of a Kafka topic.

Publish channel

The outbound side of MIF for one object structure.

Sequential / continuous queue

In order and one at a time, or in parallel with an error queue.

Server bundle

One group of Manage pods with one role: all, ui, cron, mea, report, or the stand-alone JMS server.

Built and tested on IBM Maximo Application Suite 9.2.7 with Manage 9.2.4, IoT 9.2.4 and Monitor 9.2.4 on Red Hat OpenShift, with Apache Kafka 3.9 run by the Strimzi operator 0.45, October 2026, on a demonstration cluster with demo data. Names, sizes, timings and settings are those of that one system. Product behaviour is described as observed there; the IBM and Red Hat documentation prevails.