Bootcamp / Session 1 / 1.4
Session 1.4 · Closing · Alper Kucukural
Eric showed why the plane can fly confidently in the wrong direction. Tommy showed how a good pilot flies it anyway. This last part is about the rules for flying it here, with our data, on our cluster, and what that looks like on an ordinary working day.
This part was taught in two pieces: the rules and the Foundry Connect setup before Tommy's talk, then verification and safety afterwards.
Eric ended with a table of what you will see, what causes it, and the habit that catches it. Here it is again with two more columns: what Tommy showed about each failure, and the rule or tool we use for it.
| What you see | Why (Eric, 1.2) | The pilot's habit (Tommy, 1.3) | Our rule or tool (1.4) |
|---|---|---|---|
| It makes something up | Prediction without lookup. Nothing in the mechanism checks truth. | Run generated code line by line. “Hallucination is even more dangerous” than irreproducibility. | Check every number against the raw report and every citation against the paper. A second, independent reviewer model. |
| Same prompt, different answer | Sampling: a weighted dice roll at every token. | Traceability and reproducibility are different questions. Commit to GitHub. | Log model, version, date, settings, and prompts. Keep the agent transcript and the git history. |
| Out-of-date answers | Training stops at a cutoff date. | Start with /plan and give it the context: PDFs, data, links, your old scripts. | Ground it in current documentation and your own files. A project CLAUDE.md that says what is true now. |
| Repeats the field's old mistakes | It copies the patterns in its training text, flaws included. | Foundations matter. Your experience spots the batch effect in the PCA. | Your judgement is the final verifier. Ask a domain expert before you believe a surprise. |
| Echoes sensitive text | Nothing in the mechanism keeps secrets. | Covered by the rules in section B. | Patient and unpublished data only in approved tools. Data stays on the cluster; the connector shows only what you allow. |
| Does more than you asked | Agents add tool use on top of prediction. | --permission-mode auto is fast, and it trades away the pause before each action. | Permission prompts, confirm before any write, deny rules, and hooks that enforce the rules no matter what the model decides. |
Patient data, checking outputs, disclosure, reproducibility, bias, and when to leave AI out. One rule for each, with the reason.
Rule: patient data (PHI) and unpublished results go only into tools the institution has approved for that data, with training on your inputs turned off. Never into a consumer chat account. If you are not sure a tool is approved for this data, treat it as not approved and ask.
Why: In 2023, Samsung engineers pasted confidential source code into a consumer chatbot, and the company responded by restricting the tools for everyone. Eric's FAQ answered “does it learn from my chat?” with no. Whether the provider keeps and can read what you sent is a separate question, and for patient data it is the one that matters.
Identifiers are wider than names. Dates of service, ZIP codes, record numbers, and full-face photos all count under HIPAA.
Rule: AI is never an author. Major journals, including the Nature and Science families, do not accept AI tools as authors, because a tool cannot take responsibility for the work. Every claim in your paper is yours.
Rule: name the tool and version, the task it did, and the verification you ran. Follow the venue's policy. Session 2 builds a reusable disclosure template.
We follow this rule too: see the AI use disclosure for these materials.
Rule: record the model and version, the date, settings such as temperature when you control them, and the prompts. With a coding agent, keep the session transcript and commit the code it wrote. As Tommy put it, you may not get the same output twice, but you should always be able to show how you got this one.
Rule: read AI output the way you read a field's older papers. Training text over-represents some populations, tissues, and model organisms, and the model repeats that skew fluently.
Rule: AI may do recoverable labour: formatting, first drafts of code, polishing language, summarizing a source you will check. It must not replace the judgement that defines research: framing the question, calling an outlier, interpreting a surprise, answering a reviewer, or anything clinically consequential without expert review. Leave AI out entirely when you cannot check its answer.
Tommy's demo ran an agent on a laptop against a local analysis. My day is the next step up: an agent connected to the systems this bootcamp runs on, Foundry Connect and the UMass Chan HPC cluster. In class this is a live demo; the steps below walk through it.
Every project has a CLAUDE.md: what the project is, how it is laid out, and the rules the agent follows there. I also keep a global one with rules for every project. Before any change, the agent writes a plan, and I read it before it acts.
Same idea as Tommy's “always start with a /plan”, made permanent so I do not have to repeat myself each session.
A skill is a markdown file with instructions, the SKILL.md from Tommy's slides. We write them for the procedures we do not want improvised:
A skill is a protocol written once, reviewed by a person, and followed the same way every time. That is what makes an agent's work traceable.
Through the Foundry Connect connector (the fourth green light from 1.0), I ask in plain language:
The data and the compute stay on the cluster. The assistant sees what the connector exposes, and every write waits for a person.
A chat assistant sees only what a connector shows it. A coding agent on your own machine can do anything your terminal can do. That is why permission modes matter, and why auto mode belongs in a throwaway project, not next to credentials or patient data.
A true story from this summer. While debugging a database connection, an agent helpfully printed a live access token and a database password into its session transcript. Nothing malicious happened, and nothing was misused. Both credentials still had to be revoked and replaced, because anything printed into a transcript is kept.
What changed afterwards:
CLAUDE.md: never print a credential, never type a password into a command.The model can forget or misread a rule in its instructions. A hook runs every time, whatever the model decides.
Long sessions get slower, more expensive, and less accurate. I send exploration to subagents and keep only their conclusions, and I start a fresh session once a task ships. This is Tommy's /compact and subagents slide, applied as a habit.
Adapted from Michael Zyphur's Responsible AI in Research and Research Training.
| Skill | The one-line rule | Where you saw it |
|---|---|---|
| Citation verification | Check every AI-produced citation against the primary source. | Eric's fake-reference quiz (1.2) |
| Model and parameter reporting | Report model, version, temperature, seed, and prompt. | Eric's temperature slider (1.2) |
| Prompt discipline | A prompt is an analytic choice. Test whether a small rewording changes the answer. | Session 2 and Session 3 |
| Adversarial review | Critique with a different model than the one that did the work. | Verification step (1.4 C) |
| Sycophancy detection | Agreement is not evidence. | Verification step (1.4 C) |
| Failure-mode reporting | Say what you checked, what failed, and what risk remains. | The table in section A |
Eric's quiz asked which references were invented. This one asks which prompts should never be sent to a consumer chat tool. Decide for each one before you open the card. All records below are fictional.
Here is the DESeq2 output for liver Jnk1/Jnk2 knockout versus wild type from GEO (public accession). Which of the top 50 genes are in the PPARα pathway?
Turn this into a sample description: MRN 0000-000, 52-year-old woman, vitiligo, lesional biopsy taken 03/14 at the dermatology clinic, follow-up scheduled.
Check this sample sheet for errors: sample_id, donor_age, sex, biopsy_date, home_zip, condition, batch
Make this abstract more compelling for Nature: [our unpublished preliminary result, not yet submitted]
Two minutes: whether you made it, what worked, and what to change. We ask this after every session.
Get your four green lights using the setup checklist. Everything you saw in section C depends on them, and so do Sessions 4 to 6.
Use the nine-question checklist from 1.1 and the flight rules above. Write five lines your lab could actually adopt, and bring them to Session 2. An example to react to, not to copy:
Will you be the pilot or the passenger? Pilots know how the plane works, follow the flight rules, and check the instruments against the ground.