Job workflow

Opaque jobs—whether written in code or configured using a no-code template—follow the same collaborative, secure, and auditable workflow that can be broadly broken down into four main stages:

1. Draft
2. Review
3. Execution
4. Outcome

The review process is collaborative, with different responsibilities depending on whether you're the job author or a reviewer. Differences between notebook and no-code jobs, like writing a query versus selecting a template, are noted where relevant.

1. Draft stage

In the draft phase, you define your job logic.

• For notebook jobs, you write your logic in Python (PySpark) or PySpark SQL. You can reference encrypted datasets, define input variables, and test with synthetic data—all before touching real data. Jobs remain in draft until submitted for review, allowing you to experiment freely, refine logic, and configure inputs without risk.
• For no-code jobs, you select a prewritten template and fill in the required parameters—no code needed.

Not sure which job type is right for you? See About jobs for guidance.

2. Review stage

When your job is ready, submit it for review. All workspace members must approve the job before it can run. This approval step ensures that sensitive data is never accessed or processed without unanimous member consent.

• In notebook jobs, reviewers check the query logic, any defined input variables, and the test results generated using synthetic data.
• In no-code jobs, they review the selected template and parameters.

After reviewing, members approve or reject the job. Once approved by everyone, the job can proceed to execution. If any member rejects it, the job author can cancel the review and return the job to Draft status for revision and resubmission.

3. Execution stage

Once approved, the job can be executed by the job author. Jobs run inside a secure trusted execution environment (TEE), where data is decrypted only for the duration of the computation. This ensures full confidentiality—even from infrastructure providers or system administrators.

During execution, the job query is processed within a secure enclave, and the results are returned in encrypted form. The job progresses through a few system states—Queued, Running, and then either Succeeded or Failed, depending on the outcome.

4. Outcome stage

When a job completes successfully, the results are encrypted within the secure enclave before being saved to the workspace—ensuring that no plaintext data is ever exposed. Each result is protected using the job author’s encryption keys, ensuring that only members of the author’s organization can access the data.

Results can be viewed directly on the respective job’s details page, exported for further analysis, or rerun if the job is configured to allow it. Access to results is limited to authorized users—that is, workspace members from the job author’s organization.

Next steps

• Jobs list view: Learn to navigate the jobs lists page.
• Notebook jobs: Learn how to write and test custom queries using the query editor.
• No-code jobs: Use prebuilt templates to run analyses without writing code.
• Job reruns: Configure repeatable jobs and track prior runs.