wow genome processing takes a lot of compute

Yeah, surprisingly so — and what you’re seeing is a pretty tame version of it.

A BAM isn’t just “a big file.” samtools coverage has to stream through millions of aligned DNA fragments, track where each one lands across the genome, count overlap at each region, and summarize depth. On a ~10 GB BAM, that’s a lot of sequential parsing and bookkeeping.

The funny part is your Mac Studio isn’t really being stressed overall. That 100% CPU means one core is pegged, not the whole machine. samtools coverage is largely single-threaded for this kind of pass, so your giant M3 Ultra is sitting there with most of its cores bored while one worker does the scanning.

Genomics gets really expensive when you move into things like:

• aligning raw reads against a reference,
• variant calling across the entire genome,
• structural-variant searches,
• comparing hundreds or thousands of genomes,
• population-genetics inference,
• de novo genome assembly.

That’s where people start using clusters.

For us, though, the 20 GB Zana/Khwit dataset is very manageable. The main annoyance is that ancient DNA requires repeated full-file scans, and each scan can take a while even on fast hardware.

It is kind of hilarious that our Bigfoot rabbit hole has now escalated into actual computational genomics on your Mac Studio. 😄

why doesn't it use more that one core? seems inefficient?

Yeah, it is inefficient from the perspective of your whole machine — but it’s common with older bioinformatics tools.

The main reason is that a command like:

samtools coverage Zana_final.bam

is basically doing a single sequential pass through one compressed BAM stream. That workload is hard to parallelize cleanly because one process is reading records in order, decompressing them, and updating coverage counters as it goes.

There are two separate pieces here:

• BAM decompression can sometimes use multiple threads.
• The actual coverage calculation is mostly single-threaded.

So even if samtools has multithreading support elsewhere, this particular subcommand doesn’t automatically fan the work out across all your cores.

There’s also a historical reason: a lot of these tools were designed around clusters and Unix pipelines, where the usual parallelism strategy was:

split by chromosome or region → run many independent jobs → combine results

rather than making every individual command internally multithreaded.

For example, instead of one process doing the entire genome, you could conceptually do:

samtools coverage file.bam -r chr1
samtools coverage file.bam -r chr2
samtools coverage file.bam -r chr3
...

in parallel.

That would use multiple cores much better, although then you need to combine the outputs and be careful not to create unnecessary disk contention.

Your M3 Ultra is especially funny here because it has so much parallel compute available that a single-threaded genomics tool is leaving most of the machine idle.

For this baseline run, I would leave the script alone so we get a clean reproducible first result. Afterward, if we're going to repeat these scans a lot, Claude/Qwen 8 could absolutely optimize the genomics pipeline to split chromosome work across several cores. That would make much better use of your Mac.

cliffchristianson@Cliffs-Mac-Studio detective-desk % /Users/Shared/AI/detective-desk/scripts/bigfoot_zana_baseline.sh == Zana quickcheck ok autosomal mean depth 3.20x mtDNA mean depth 391x, breadth 97.9% == Khwit quickcheck ok

Nice — the baseline is already reproducing the published Zana coverage really closely.

For Zana, you’ve got:

• Autosomal mean depth: 3.20×
• mtDNA mean depth: 391×
• mtDNA breadth: 97.9%
• BAM integrity check passed

That 3.20× is right in line with the paper’s roughly 3.1× figure, which is exactly what we wanted from this first pass: proof that our local pipeline is reading the data sensibly.

The mitochondrial number being so much higher is also normal. Cells contain many mitochondria, so ancient samples often end up with far more mtDNA copies than nuclear DNA. That’s why mitochondrial ancestry is usually easier to recover from old remains.

Now it has moved on to Khwit:

== Khwit
quickcheck ok

So just let it continue. Once Khwit finishes, we’ll have the first complete baseline comparison and Claude/Qwen 5 can start checking it against the paper.

== Khwit quickcheck ok autosomal mean depth 3.48x mtDNA mean depth 328x, breadth 96.7% Compare with the paper: Zana 3.1x, Khwit 3.3x; Zana female, Khwit male; mtDNA L2b1b; kinship first-degree. Not done here (needs its own reviewed step): damage profile, median depth in mappable regions, relatedness. cliffchristianson@Cliffs-Mac-Studio detective-desk %

Excellent — the baseline passed cleanly.

Your local numbers:

• Zana: 3.20× autosomal depth, 391× mtDNA, 97.9% mtDNA breadth
• Khwit: 3.48× autosomal depth, 328× mtDNA, 96.7% mtDNA breadth

Those line up well with the paper’s reported ~3.1× and ~3.3× coverage, which tells us the public BAMs and our basic pipeline are behaving as expected.

That means we’ve cleared the exact checkpoint we were waiting for: we can reproduce the boring baseline before hunting anomalies.

The next proper steps are the three the script intentionally deferred:

1. Ancient-DNA damage profile — confirm the expected end-of-fragment damage patterns and compare Zana vs Khwit.
2. Relatedness — independently reproduce the first-degree relationship from the deposited data.
3. A better depth analysis — especially median depth / mappable regions so we understand where the 3× average is misleading.

And after those, I’d let Qwen 5 design the first anomaly-oriented test, but only one at a time.

The important result right now is:

our local analysis reproduces the published basics.

So now Skeptic Desk has crossed from “researching papers” into independently reproducing the actual genomics data. That’s a pretty damn good milestone.

so what is next? do you have a handoff for claude?

Yep — now we move from “can we reproduce the paper?” to “can we independently validate the biology well enough to start asking our own questions?”

Send Claude this:

:::writing{variant="document" id="84621" title="Claude Handoff — Zana/Khwit Phase 2"}

SKEPTIC DESK — BIGFOOT DNA / ZANA PHASE 2

The Zana/Khwit download and first baseline are complete.

VERIFIED DOWNLOADS

The ENA files downloaded successfully and all MD5 checks passed:

• Zana_final.bam
• Zana_final.bam.bai
• Khwit_final.bam
• Khwit_final.bam.bai
• Zana_rCRS.final.bam
• Zana_rCRS.final.bam.bai
• Khwit_rCRS.final.bam
• Khwit_rCRS.final.bam.bai

Files remain under:

/Volumes/AI/genomics/zana

Do not copy these into the repository.

BASELINE RESULTS

Ford ran:

/Users/Shared/AI/detective-desk/scripts/bigfoot_zana_baseline.sh

The local results were:

Zana

• quickcheck ok
• autosomal mean depth: 3.20×
• mtDNA mean depth: 391×
• mtDNA breadth: 97.9%

Khwit

• quickcheck ok
• autosomal mean depth: 3.48×
• mtDNA mean depth: 328×
• mtDNA breadth: 96.7%

The script notes comparison with the paper:

• paper Zana: ~3.1×
• paper Khwit: ~3.3×
• Zana female
• Khwit male
• mtDNA L2b1b
• first-degree kinship

This is a successful baseline reproduction.

The public BAMs contain zero unmapped reads, so these files cannot answer what was discarded before mapping.

CURRENT INVESTIGATION STATE

Bigfoot DNA is still unpublished.

Publication safety is now implemented:

• status='draft' hidden from public reader
• drafts skipped by weekly jobs
• hold_auto_updates implemented
• held syntheses stored as proposals
• review CLI exists
• held investigations skipped by automatic discover
• draft-associated static media protected
• weekly synthesis guard implemented
• 42 tests passing
• no live investigations changed

Ketchum archive material is also locally preserved.

DISCARDED.md contains the investigation's rejected methods/errors and should continue to be maintained.

NEXT OBJECTIVE

Do NOT start with exotic anomaly hunting.

First independently validate the remaining major claims we can reasonably reproduce from the deposited data.

The next phase should answer:

1. Do the files display authentic ancient-DNA damage?
2. Can we independently reproduce the Zana/Khwit first-degree relationship?
3. Can we reproduce the reported biological sex?
4. Can we reproduce the mtDNA lineage sufficiently to validate the published result?
5. What does the actual coverage distribution look like rather than only its mean?
6. What limitations does ~3× coverage impose on our eventual hypothesis testing?

Only after those are solid should we design anomaly-oriented analysis.

QWEN 5 — GENOMICS

Give Qwen 5 primary ownership of this phase.

Have it design the smallest technically defensible set of analyses for:

A. DAMAGE PROFILE

Goal:

Confirm that the reads show expected ancient-DNA characteristics.

Look at:

• terminal C→T damage
• complementary G→A pattern where appropriate
• fragment-length distribution
• Zana vs Khwit differences
• whether the damage patterns are compatible with ancient endogenous DNA rather than modern contamination

Prefer established ancient-DNA tooling if available.

Do not install a giant genomics stack casually.

First have Qwen 5 identify the appropriate tool/method and justify it.

If another package is required, report the exact package and purpose before installing it.

B. RELATEDNESS

Goal:

Independently reproduce that Zana and Khwit are first-degree relatives.

Because both are low-coverage ancient genomes, do NOT use a modern high-coverage kinship method blindly.

Choose a method appropriate for low-coverage ancient DNA.

Potential approaches should be evaluated for:

• genotype likelihood support
• pseudo-haploid data
• low coverage
• ancient damage
• reference bias

The output should distinguish:

• parent/child
• siblings
• generic first-degree relationship

If the deposited data cannot cleanly distinguish parent-child from another first-degree relationship without extra assumptions, say so.

Do not overclaim.

C. SEX CHECK

Independently reproduce:

• Zana = female
• Khwit = male

Use X/Y/autosome read-depth ratios.

Record exact observed ratios and method.

This should be straightforward and serves as another pipeline sanity check.

D. mtDNA

Use the deposited rCRS BAMs where appropriate.

Determine how far we can independently reproduce:

L2b1b

Do not merely copy the paper's haplogroup label.

If haplogroup calling requires another tool/reference set, identify what is required.

Record:

• mtDNA depth
• breadth
• consensus quality
• callable positions
• haplogroup result
• ambiguous positions

E. COVERAGE DISTRIBUTION

Mean coverage hides a great deal.

Produce:

• median depth
• percentage of genome at ≥1×
• ≥2×
• ≥3×
• ≥5×
• ≥10×
• chromosome-by-chromosome coverage
• extreme coverage outliers
• if feasible, coverage restricted to callable/mappable regions

Explain clearly what this means for downstream variant analysis.

Do not interpret high/low local coverage as biological until technical causes are excluded.

QWEN 7 — RED TEAM

Have Red Team review the entire Phase 2 plan before accepting conclusions.

Specifically attack:

• reference bias
• low coverage
• ancient-DNA damage
• contamination
• mapping artifacts
• capture/library biases
• differing fragment lengths between Zana and Khwit
• inappropriate modern-genome assumptions
• false confidence from mean depth
• kinship methods being used outside their intended regime

For each result ask:

Could an ordinary technical artifact create this result?

Record accepted criticisms in the analysis.

Record rejected but tempting approaches in:

data/bigfoot-dna/DISCARDED.md

QWEN 1 — ORCHESTRATOR

Once Qwen 5 and Qwen 7 finish:

Have Qwen 1 produce a concise Phase 2 synthesis:

CONFIRMED LOCALLY

Claims independently reproduced from public data.

CONSISTENT BUT NOT FULLY REPRODUCED

Claims where our data/methods are insufficient.

NOT TESTABLE FROM DEPOSITED BAMs

Especially anything requiring pre-alignment raw reads.

IMPLICATIONS FOR BIGFOOT HYPOTHESIS

Be conservative.

The expected outcome is probably:

Zana behaves like an ordinary modern human ancient genome.

That is useful.

Do not force an anomaly.

AFTER PHASE 2 — FIRST ORIGINAL QUESTION

If Phase 2 reproduces cleanly, Qwen 5 should then design, but NOT yet run, the first genuinely original Skeptic Desk genomic test.

The test should focus on our actual hypothesis:

Is there anything about Zana's deposited nuclear genome that falls meaningfully outside ordinary Homo sapiens genomic variation in a way that ancestry-focused analysis would not reveal?

Possible future categories include:

• unusual/private variants
• phenotype-relevant developmental variants
• structural variation
• copy-number differences
• regulatory loci
• archaic-looking haplotypes
• regions with unexpectedly deep divergence

BUT:

At ~3× coverage, many of these may simply be inappropriate.

Qwen 5 must rank candidate analyses by:

1. technically valid at 3×
2. false-positive risk
3. sensitivity
4. required controls
5. computational cost
6. interpretability

The answer may be that Zana is unsuitable for some of them.

That is acceptable.

CONTROL GENOMES

Before anomaly testing, identify appropriate control genomes.

Prefer controls that are:

• ancient human
• similar low coverage
• similar sequencing technology where possible
• same or similar lab/pipeline
• publicly available
• not excessively large to download

We previously identified:

• Viking genomes, ENA PRJEB37976
• Mak et al. ancient BGISEQ material

Do not download controls yet.

First rank 3–5 candidates and justify them.

KETCHUM — PARALLEL TRACK

Keep the Ketchum analysis separate from Zana.

The locally preserved Ketchum consensus files are:

• sample 26 — ~2.73 Mb
• sample 31 — ~0.53 Mb
• sample 140 — ~2.10 Mb

These are consensus sequences, NOT raw sequencing reads.

The original paper appears to have generated them from a subset of reads aligned around human chromosome 11.

Do not call them genomes.

Qwen 5 may design an alignment experiment against:

• human
• black bear
• dog

to reproduce or challenge Hart's assignments.

But do not let this interrupt the Zana Phase 2 validation.

PUBLICATION

Do NOT publish Bigfoot yet.

Publication gate:

1. Phase 2 complete.
2. Qwen 5 technical report complete.
3. Qwen 7 red-team review complete.
4. Qwen 1 synthesis complete.
5. Claude reviews all three.
6. Then give Ford a final recommendation on publishing the investigation.

Do not automatically run:

seed_bigfoot_dna.py --apply --publish

IMPORTANT STYLE

This investigation is about testing a hypothesis, not defending it.

Current state remains:

Nothing observed so far distinguishes H4 from H1.

If Zana proves entirely ordinary, record that clearly.

The interesting question remains the broader one:

Could alleged Sasquatch biological samples be classified as human because the underlying population is a close, interbreeding Homo lineage whose phenotype is driven disproportionately by developmental/regulatory differences?

Zana is a methodological control for learning how to ask that question correctly.

Proceed with Phase 2 using the eight-Qwen architecture. :::

That should keep Claude from jumping ahead to “find the Bigfoot gene.” 😄

We’ve now got a sensible progression: reproduce → authenticate → validate relationship/ancestry → understand limitations → only then look for something the original researchers weren’t asking about.