Design Pipeline
User Guide — Build, Version, and Run Your Data Pipelines
| Last updated: September 2026
Contents
- Overview
- Finding Your Way Around
- Creating a New Pipeline
- Building the Pipeline
- Adding and Configuring Steps
- The Pipeline Action Buttons
- Running a Pipeline
- Quick Reference
1. Overview
Design Pipeline is the visual builder for your data pipelines — the jobs that move and transform data from one place to another. You lay a pipeline out on a canvas by dragging in the steps you need, connect them into a flow, set each one up, and then run it — all from one screen.
It's a drag-and-drop workbench: the tools you can use sit in a panel on the right, your saved pipelines on the left, and the canvas in the middle is where you assemble them. When a pipeline is ready you run it, then follow its progress in Process Monitor.
What you can do here
- Create a pipeline from a ready-made template.
- Drag in steps and arrange them into a flow on the canvas.
- Configure each step to do exactly what you need.
- Keep versions of a pipeline and look back through its history.
- Run a pipeline for a chosen date and hand off to Process Monitor to watch it.
WHERE TO FIND IT — Open Management → Design Pipeline.
WHAT YOU CAN DO DEPENDS ON YOUR ACCESS — Creating, editing, versioning, and deleting are each controlled by permissions. If a button described here doesn't appear for you, your access level doesn't include it — you'll still be able to open and view pipelines.
2. Finding Your Way Around
The screen is arranged in three areas, left to right:
- Pipelines panel (far left) — a + Create New Pipeline button, a Search box, and the list of your saved pipelines. Select one to open it on the canvas.
- The canvas (centre) — the name of the open pipeline and a row of action buttons across the top, then the building area below where the pipeline's steps are laid out, and a settings area beneath that for the selected step.
- Tools panel (far right) — the palette of steps (tools) you can add, based on the pipeline's template. You drag these onto the canvas.
The usual flow is: pick or create a pipeline on the left, drag steps from the right onto the canvas, set each one up in the area below, then run it from the top.
FIND A PIPELINE FAST — Use the Search box above the pipelines list to filter by name when the list gets long.
3. Creating a New Pipeline
Every pipeline starts from a template — a ready-made starting point for a common kind of job. There's no blank option; you pick the template closest to what you need and adjust it from there.
Steps
- In the left panel, click + Create New Pipeline. A Create New Pipeline panel opens.
- Enter a Pipeline Name.
- Under "What do you want to start with?", choose a template from the list. Each option is one of the templates set up for your system.
- Click Get Started →. (The button stays switched off until you've entered a name and picked a template.)
Your new pipeline opens on the canvas, ready to build. The template you choose also decides which tools appear in the right-hand panel for this pipeline.
THE TEMPLATE SETS YOUR TOOLBOX — Because the template determines the steps available on the right, pick the one that matches the job you're doing. You can still add, remove, and rearrange steps afterwards.
4. Building the Pipeline
You build a pipeline by placing steps on the canvas and arranging them into the order they should run. (Steps are also called tools or tasks.)
- Add a step — drag a tool from the right-hand Tools panel onto the canvas. It becomes a step where you drop it. (See Adding and Configuring Steps for how the Tools panel works and how to set each step up.)
- Move a step — drag a step around the canvas to arrange the flow.
- Remove a step — right-click the step and choose Delete.
- Rename the pipeline — double-click the pipeline name at the top, type the new name, then click away. The name saves on its own.
Across the top of the canvas is a row of action buttons for the whole pipeline — Create Version, History, Properties, and Delete. These are explained in The Pipeline Action Buttons.
5. Adding and Configuring Steps
The Tools panel on the right is your box of building blocks, and the settings area below the canvas is where you set each block up. This section covers both, and how to configure every kind of step.
5.1 The Tools panel
The right-hand panel lists the steps available for this pipeline's template, each shown with an icon and a name. Which tools you see depends on your template — you won't always have all of them. Not every tool is usable at every moment, either:
- Available tools are highlighted and can be dragged onto the canvas.
- Tools that don't apply yet appear dimmed and can't be dragged — usually because an earlier step needs to be in place first.
5.2 Adding and selecting a step
- Drag an available tool from the Tools panel onto the canvas — it becomes a step where you drop it.
- Click the step to select it. Its settings open in the area below the canvas.
- The settings are matched to the kind of step — set them up using the matching section below, then move on to the next step.
5.3 The three ways a step is set up
There are many tools, but they're configured in one of three ways. Find your tool in this table, then follow the section it points to.
| Setup type | Tools that use it | How to configure |
|---|---|---|
| Source setup | Source, Data Processing | 5.4 |
| Table setup | Landing Zone, Transient Layer, Conform Data, Data Mart, Aggregated Data, Publish Target | 5.5 |
| Mapping setup | Extract, Transform, Confirm, Aggregation, Publish Mapping | 5.6 |
5.4 Configuring a Source step
A Source step (and the general Data Processing step) reads data in. Its settings have three tabs — work through them left to right:
- Task — give the step a Task Name so you can recognise it on the canvas.
- Source — choose the Source to read from, from the dropdown. Its Alias, Description, and Type are shown so you can confirm you've picked the right one. To set up a source that isn't listed, use New.
- Connection — review the connection details for the source you chose. This tab shows how OnCoor connects to it; there's nothing to fill in unless you're changing the connection.
5.5 Configuring a table step
The table steps — Landing Zone, Transient Layer, Conform Data, Data Mart, Aggregated Data, and Publish Target — each point at a table that data is written to or read from. Their settings have two tabs:
- Task — give the step a Task Name.
- Table Property — choose the table by its Fully Qualified Name. Its Schema Name and Table Name fill in automatically once you pick it. Set IsHeader to Yes if this is the header table for the record.
The tables you choose here are the ones you define in the Data Model: Landing Zone steps use Landing Zone tables, Transient Layer steps use Transient Layer tables, and Conform Data steps use Conform Data tables.
5.6 Configuring a mapping step
The mapping steps — Extract, Transform, Confirm, Aggregation, and Publish Mapping — apply a map that turns one table into another. Each is set up in a single form:
- Choose the Map Name (for example, Extract Map Name) from the dropdown. To build a new map, use New — it opens the mapping editor already pointed at the right tables.
- The Source Table and Target Table fill in automatically from the map you chose — you don't type them.
- Check they line up. The source and target are matched against the steps immediately before and after this one in the pipeline. If they don't match, a "do not match" warning appears — choose a different map, or adjust the neighbouring steps, so the flow connects correctly.
SET EACH STEP BEFORE YOU RUN — A step you've dragged in but not configured won't do the right thing yet. Work through the pipeline one step at a time, completing each step's setup, so the whole flow is ready before you run it.
SELECT, THEN SET UP — A step you've dragged in but not configured won't do the right thing yet. Click each step and complete its settings below the canvas before running the pipeline.
6. The Pipeline Action Buttons
The buttons across the top of the canvas act on the whole pipeline. Here's what each one does.
| Button | What it opens | What you do there |
|---|---|---|
| Create Version | Add New Pipeline Version | Save the current pipeline as a new version. You set an Effective Date and Expiry Date for it. The earlier version stays as it was. |
| History | Pipeline Versions | See a table of all the pipeline's versions, with their effective and expiry dates and run settings. On each row you can Select a version to make it the one you're working on, or Delete a version you no longer need. |
| Properties | Pipeline Properties | View and adjust the pipeline's overall settings — its Name and Description, run-schedule options (such as Support Monthly, Support Daily, and the month-end day settings), and load options (such as Ignore As Of Date and Support Incremental Load), along with the effective and expiry dates. |
| Delete | Confirmation prompt | Delete the pipeline. This is available when the pipeline has a single version. |
About versions. A pipeline can have several versions, so you can change it safely and keep a record of what it looked like before. Use Create Version before a big change, and History to review earlier versions or switch back to one. Each version carries its own Effective Date and Expiry Date, which control the period it applies to.
VERSION BEFORE A BIG CHANGE — Creating a version first means you can always go back. History is the safety net for your pipeline design.
7. Running a Pipeline
When a pipeline is built and configured, you run it from the top of the canvas.
Steps
- Click Execute.
- In the dialog, choose an As Of Date — the reporting date you want the run to be for.
- Click Run Pipeline.
The pipeline starts running. To watch its progress — step by step, with timings, record counts, and any errors — click Process Monitor at the top of the canvas, or open Process Monitor directly. Your run appears there under User Process.
RUN, THEN MONITOR — Design Pipeline is where you build and start a run; Process Monitor is where you watch it happen. The Process Monitor button takes you straight there.
8. Quick Reference
A fast lookup for the most common actions.
| I want to… | Do this |
|---|---|
| Open Design Pipeline | Management → Design Pipeline |
| Open a saved pipeline | Select it in the left-hand Pipelines list |
| Find a pipeline by name | Type in the Search box above the list |
| Create a pipeline | + Create New Pipeline → name it → choose a template → Get Started → |
| Add a step | Drag an available tool from the right-hand Tools panel onto the canvas |
| Rearrange steps | Drag them around the canvas |
| Remove a step | Right-click the step → Delete |
| Set up a step | Select it → fill in its settings in the area below the canvas |
| Rename the pipeline | Double-click the name → type → click away |
| Save a new version | Create Version → set Effective and Expiry dates |
| Review or switch versions | History → Select or Delete a version |
| Change pipeline-wide settings | Properties |
| Run the pipeline | Execute → pick As Of Date → Run Pipeline |
| Watch a run | Process Monitor (top of the canvas) → User Process |
Source: OnCoor Management module product documentation, written for end users. For the latest screens and options, always refer to the in-app interface.