feat(captain-data): add piece with actions to schedule a workflow and retrieve results

[AdamSelene]

What does this PR do?

Add new integration with Captain Data (API doc)

• schedule a workflow
• retrieve results
• custom API call

[nx-cloud]

☁️ Nx Cloud Report

CI is running/has finished running commands for commit 5a6321e. As they complete they will appear below. Click to see the status, the terminal output, and the build insights.

📂 See all runs for this CI Pipeline Execution

---

✅ Successfully ran 3 targets

• nx affected --target=build -c production --parallel=3
• nx affected --target=lint --parallel=3
• nx run-many --target=test --projects=engine,shared,server-api --parallel=3

---

Sent with 💌 from NxCloud.

[AdamSelene]

Logo:

[kishanprmr]

Is it possible to create separate field for each workflow step configuration instead of JSON?

[AdamSelene]

In theory yes but it would require more work to generate the dynamic props - I'd rather do it in a second stage (particularly since I'm myself not very familiar with Captain Data)
My specific use case here is to prevent my users from using a generic HTTP pieces with unsafe API keys in headers 😬

[kishanprmr]

We can add this action in the second stage, as it will make it easier for the user to set up the workflow configuration from dynamic properties. What do you think?

[AdamSelene]

You mean remove the action for now and add it back in a second PR when it's more "baked"? I can do that yes!
As long as we have the custom api call users won't be blocked

[kishanprmr]

Yes, please add this action in the next PR. Let's merge it after you remove it.

[AdamSelene]

@kishanprmr I've spent a few hours trying to have something nice with dynamic props, but it turns out it's not possible today - I would need to nest dynamic drop-downs and objects and arrays within an array and ActivePieces does not allow it
I ranted a bit on our shared Slack channel 😬

Bottom line: either we ship the action with a several JSON fields (inputs and steps at least), or we drop it and leave users with custom api call.
I think it's still better to have the explicit action, even if not perfect - WDYT?

{
    "job_name": "My Linkedin Company Search job ", -> ShortText, easy
    "unstructured_meta": true, -> Checkbox but effect of "true" is not documented
    "inputs": [ -> Array but why? Should be a single object :face_palm: Can't have nested object in array yet
        { 
            "linkedin_company_search_url": "https://www.linkedin.com/search/results/companies/mySearch"        
        }
    ],
    "steps": [{ -> this gets wild
        "accounts": ["40a0b02-1d2e-406e-a2b7-ad3d83948264"], // A Linkedin account UID - but there is no way to tell from the API that for this specific STEP of this specific WORKFLOW you need a LINKEDIN account and not KASPR or LUSHA or GITHUB or any of the 29 available integrations
        "parameters": {"max_results": 42}, // Can't nest an Object in an Array
        "step_uid": "f34de6f7-da13-4dc2-bfdb-27601e651be4", // By wrapping the Array within a DynamicProperties I can use a StaticDropdown here
    }],
}