Each submission meant someone retyping the same details into yet another web portal. About 30 minutes a time.
The problem
Our teams submit to dozens of external portals, and no two of them behave alike. Fields, page flows, upload rules and verification steps all change from site to site. If I'd written one bot per portal, the codebase would've grown with every new site and broken whenever one got redesigned.
The data didn't help either. Our CRM keeps a full name in one field, countries as codes and dates in one format, and every portal wants something different.
- 1PickAn operator picks the record and the portalOne click from a browser side panel.
- 2PrepareThe data is cleaned and mappedRaw CRM values are reshaped for this portal. Anything missing is flagged before the bot starts.
- 3FillThe bot fills the form, page by pageIt follows the portal's config, skips questions that don't apply and retries if a page is slow.
- 4Hand overA person steps in only when neededOTPs, CAPTCHAs and uploads. The bot waits, then picks up where it stopped.
- 5DoneSubmitted and loggedEvery step's outcome is recorded against the job and the team.
Calls I made
- New portals come in as configuration.A new portal is added as config, in a workflow editor we built. Coverage grew to 71 portals while the codebase stayed the same size, and config changes never needed a redeploy.
- Clean the data once, before the bot starts.A Python step turns raw CRM data (names, dates, country codes, phone numbers, yes/no answers) into portal-ready values. The bot never touches messy data, and missing fields show up before a run, not halfway through it.
- Let the page decide which questions apply.Portals show and hide questions based on earlier answers. Rules in the config (if this is on the page, if that step worked, repeat until the list runs out) check what's really there and skip the rest, so one config covers every branch of a form.
- I designed the human steps in from day one.OTPs and CAPTCHAs go to a person, and the bot picks up again once they're done. Trying to automate those steps would have made the whole thing fragile. Small failures, like a slow page, retry on their own with a growing wait.
- Every failure gets a label.When a run fails, the platform reads the error and tags it: a config problem, a portal that changed, a connection issue, or something that needs a person. The fix goes to the right person on the first try, not after a round of guessing.
- I defined success so it can't flatter us.Success rate is successful jobs divided by successful plus failed. If a user stops a job on purpose, it doesn't count against the platform.
By hand it's about 30 minutes. The bot's own time averages about 5 minutes per job. Figures as of October 2026.
