# Full Text: 2026_CrossTalk_on_the_Page

> Extracted from `CrossTalk-on-the-Page-v0.1.0.pdf`

---

## Page 1

CrossTalk on the Page
How defense software engineering was written and illustrated
Daniel Ari Friedman
Active Inference Institute
ORCID: 0000-0001-6232-9096
DOI: 10.5281/zenodo.24297224
2026-11-16 · Public edition 0.1.0
CROSSTALK / ISSUE FRONTS
The retained issues by year
One drawn glyph for each of 281 retained issue PDFs, 1989-2025.
1989
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
2025
Cover
Newsletter front
Separate publisher image
Original scope open

## Page 2

CROSSTALK / 02
GRAPHICAL ABSTRACT
GRAPHICAL ABSTRACT / TEXT
How text becomes searchable
Native text, optical character recognition (OCR) and pages with neither.
Composition of each retained PDF
Equal-width columns follow retained-issue order, including combined issues.
0%
25%
50%
75%
100%
1989
1995
2000
2005
2010
2015
2020
2025
Page-weighted totals by era
1989-1994
1,154 pages
1995-1999
1,921 pages
2000-2009
3,839 pages
2010-2019
1,888 pages
2020-2025
858 pages
7,102 native (73.5%)
2,539 OCR only (26.3%)
19 without text (0.2%)
Admitted text can be as short as a running header. It does not establish article completeness.
Thin columns show each retained PDF in chronological issue order. The era bars give each physical page equal weight. The two views
distinguish native text, OCR-only text and no admitted text.

## Page 3

CROSSTALK / 03
GRAPHICAL ABSTRACT
GRAPHICAL ABSTRACT / PAGE APPEARANCE
Four patterns of page layout
Each layout map pools members of one group across 9,660 pages.
Mean darkness in a common page frame
Coarse cells summarize each group's aggregate map. All four use the same grayscale.
GROUP 01
8,865 pages
92.8% issue-weighted
281 issues present
GROUP 02
305 pages
2.6% issue-weighted
179 issues present
GROUP 03
282 pages
2.6% issue-weighted
145 issues present
GROUP 04
208 pages
1.9% issue-weighted
150 issues present
Each issue contributes the same total weight. Group numbers run from lightest to darkest.
Five refits held out whole issues. Adjusted Rand agreement on held-out pages was 0.99-1.00.
These patterns describe retained page appearance, including digitization and scan effects.
These common-scale maps show twelve by seventeen cells from each study-authored aggregate raster. Group sizes and issue-weighted
shares describe retained page appearance.

## Page 4

CROSSTALK / 04
GRAPHICAL ABSTRACT
GRAPHICAL ABSTRACT / COVERAGE
What the evidence covers
Page measurement, full-article reading and exact censuses have different units.
Pages within the retained material
Visual measurement
9,660 / 9,660  (100.0%)
Admitted text
9,641 / 9,660  (99.8%)
Direct reach of complete article appraisals
8,160 / 9,660  (84.5%)
Exact article censuses within issue sets
Retained issue scopes
270 / 281  (96.1%)
Originally open priority set
148 / 158  (93.7%)
2,708
Distinct complete appraisals
AI-assisted full-body reading
11
Original print scopes open
197 retained pages / 66 confirmed bodies
Historical totals for issues and original print articles remain unknown.
Each bar names its denominator. Page measurements, exact issue censuses and the distinct complete-appraisal register answer different
coverage questions.

## Page 5

Contents
Abstract
5
1
Reading the journal as a record of practice
5
1.1
Research questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5
1.2
Scope and contribution
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5
1.3
Status of this manuscript
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6
2
How the archive was built and checked
6
2.1
Frame, design and chronology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6
2.2
Acquisition and source custody
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6
2.3
Text units and recovery
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6
2.4
Article discovery and review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7
2.5
Language and statistical summaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7
2.6
Visual measurements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
8
2.6.1
Reconcile a figure across editions
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
8
2.6.2
Audit the boundaries of surviving bodies
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
8
3
Results
9
3.1
Archive coverage and article evidence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
9
3.2
Language depends on what the archive makes readable . . . . . . . . . . . . . . . . . . . . . . . . . . . .
9
3.3
Page structure and publication form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
9
3.4
What complete article readings add . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
10
3.5
What the publisher archived . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
10
3.5.1
Date signals and related publication copies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
11
3.5.2
Follow the folio, then the text . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
11
3.5.3
One error cohort survives in a second carrier
. . . . . . . . . . . . . . . . . . . . . . . . . . . . .
11
3.5.4
Catalogs narrow the search for paper
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
11
3.5.5
What survives in the eleven open issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
11
3.5.6
Where prose and evidence diverge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
12
3.5.7
Tracing the priority census . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
12
3.5.8
What an original copy must establish . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
12
3.5.9
Auditing continuity in source text and images . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
12
3.5.10 How repeated observations change the evidence
. . . . . . . . . . . . . . . . . . . . . . . . . . .
13
3.5.11 A reengineering assessment across two publications . . . . . . . . . . . . . . . . . . . . . . . . . .
13
4
What the surviving record can support
13
4.1
Limitations and future verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
13
4.2
Research artifacts and access
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
13
5
Figures
14
5.1
Native sensitivity and historical OCR contrasts
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
14
5.2
The archive changes the topic map
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
15
5.3
Which words carry the topic map? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
16
5.4
Do front-page patterns track the words inside? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
17
5.5
Where full reading reaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
18
5.6
A preview, its publisher links, and the full article
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
19
5.7
The publisher directory and feed together index the surviving run
. . . . . . . . . . . . . . . . . . . . . .
20
5.8
How the PASS error count fell . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
21
5.9
Pages and articles in eleven open issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
22
5.10 Exact wording across two 1992 versions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
23
6
Appendix. Source and evidence units
24
4

## Page 6

Abstract
CrossTalk offers a longitudinal view of how defense software engineering practitioners present methods, evidence and insti-
tutional priorities. We built a source-linked archive of 281 retained issue PDFs and 9660 physical pages. Native text and
admitted optical extraction make 9641 positions searchable, and page appearance is measured throughout the retained frame.
Exact article counts are available for 270 retained issues under their attested scopes, including 148 of 158 initially unresolved
PDFs. The register contains 2708 distinct complete AI-assisted appraisals. Contrasts in vocabulary and page geometry show
how strongly some apparent historical changes depend on the extraction layer, issue weighting and editorial layout. Print
and publisher-web comparisons also expose substantive work surviving outside the issue PDFs. Across 11 files with open
original scopes we examined all 197 retained pages and counted 66 confirmed complete bodies. The results describe the
acquired sources and leave the historical issue and article denominators open.
1
Reading the journal as a record of practice
The 1968 NATO conference on software engineering treated design, production, and service as linked problems and recorded
disagreement about how to organize them [Naur and Randell, 1969]. Brooks later distinguished the work of specifying and
designing a complex system from work caused by its tools and representation [Brooks, 1987]. Defense acquisition added
a related tension. The Defense Science Board’s 1994 report on commercial software urged changes to architecture, early
engineering, and iterative development to make procurement and fielding more adaptable [Defense Science Board, 1994].
The persistence of these concerns makes it possible to ask a historical question about CrossTalk. When authors describe a
recurring diﬀiculty, do they change the proposed remedy, the evidence they offer, or the language they use to make the case?
CrossTalk identifies itself as a journal of defense software engineering [cro]. The Naval Postgraduate School describes the
current journal as open access and peer reviewed. Historical issues supply their own evidence about editorial roles and review
procedures. Its issues carry technical features, sponsor editorials, letters, practitioner reports, and BackTalk. Each genre
provides a different occasion for a claim. A signed editorial can establish what its sponsor advocated, a field report can
document what a team says it tried, and a comparative study can estimate an effect when its design supports the inference.
Genre scholarship connects recurrent forms and recognized communicative purposes [Orlikowski and Yates, 1994]. Here the
distinctions guide both the review of claims and the interpretation of page patterns. A technical diagram, recurring masthead,
and cover illustration can occupy similar areas of a page while doing different work.
1.1
Research questions
1. Which problems, proposals, and forms of evidence recur in the retained journal?
2. Where do authors report observed outcomes, and what would be needed to compare their results across settings?
3. How do engineering vocabulary and the language of risk, capability, and obligation vary with period, genre, and text
availability?
4. How do page layouts and visual objects communicate a technical argument, present evidence, or mark an editorial
role?
5. Which differences survive explicit changes in issue weighting, coverage, extraction layer, and page eligibility?
An increase in a security phrase can follow a themed issue, a new editorial label, or a shift in spelling and terminology. A
change in visual density can follow production technology, scan quality, or a different mix of front and interior pages. The
study treats these mechanisms as competing explanations and uses exact issue and page keys to inspect cases. SWEBOK
provides a contemporary reference for the names of engineering domains [Washizaki, 2024]. The domain names serve as
retrieval aids across the historical material. Period-specific usage and article-level meaning require inspection.
1.2
Scope and contribution
The journal website and its Calhoun series define the initial collection, supplemented by independently reconciled holdings
[cro, nps]. The historical denominator remains unresolved. Petersen’s twentieth-anniversary account dates first circulation
to 22 July 1988 and describes artificial intelligence as an early subject [Petersen, 2008]. The account is later testimony,
inspected on printed page 13 (physical PDF page 14). It gives an explicit reason to test any apparent first occurrence
against gaps in early holdings and text retrieval.
The project follows the distinction between systematic mapping and deeper review in software engineering evidence synthesis
[Petersen et al., 2015]. Its journal-bounded map enumerates sources, physical pages, article boundaries where adjudicated,
5

## Page 7

and retrieval evidence. Its full-text appraisals identify what selected contributions claim, the kind of support they offer, and
limits on inference. PRISMA informs documentation of identification, exclusions, and eligible units [Page et al., 2021]. The
data statement makes corpus, language, and acquisition constraints explicit [Bender and Friedman, 2018].
1.3
Status of this manuscript
All 259 captured NPS issue PDFs have verified current native extraction covering 8,597 physical pages. The study also
incorporates separately sourced issues, admitted OCR, whole-page appearance measurements, two-view native category
retrieval, and thirteen source-linked physical-page observations supporting eight bounded claims. The current article and
reading ledgers report the distinct denominators for the retained corpus.
The archival map, text measures, and visual
measurements remain separately indexed so the availability of one layer cannot fill a gap in another.
The corpus analyses are exploratory. Development used previously inspected material, and independent human appraisal and
transcription evaluation remain open. The new issue-balanced topic profiles measure literal language and support source
retrieval, in keeping with problem-specific validation for text analysis [Grimmer and Stewart, 2013]. The following sections
describe the acquisition and review methods, the study findings, and the remaining uncertainties.
2
How the archive was built and checked
2.1
Frame, design and chronology
This is an exploratory literature and multimodal atlas of an acquired publication frame. The retained NPS series, independent
publisher listings, actual PDF bytes, article observations and reviewed interpretations are separate layers. A record in a
repository does not establish a historically complete issue run. A listed article does not establish a reviewed contribution.
Every result identifies its eligible frame and its unit.
The expanded source selections and analysis settings were developed after corpus inspection. Their recorded seeds and input
digests make them reproducible, not preregistered. PRISMA-P and PRISMA 2020 inform transparent selection accounting;
they do not supply a quality score or certify that a systematic review is complete [Moher et al., 2015, Page et al., 2021].
The protocol and dated amendments preserve inspected development material and the conditions for a future untouched
validation sample.
Repository issued dates, dates parsed from titles, printed publication dates and archive capture dates retain their own fields.
Combined issues, supplements, reprints, alternate scans and unresolved identities are not filled into an assumed monthly
calendar. Historical reconciliation uses explicit publisher assertions and contemporary schedule evidence. Exact PDF hashes
identify duplicate bytes; similar content only proposes an identity match for review.
2.2
Acquisition and source custody
Acquisition is bounded and records the observed locator, source family, request policy, response, byte count, digest and
outcome. Existing verified payloads are rehashed and reused. New requests use the applicable host policy and spacing;
redirects require a recorded, reviewed relation.
Successful peers remain retained when another source fails.
Prepared,
completed and unresolved attempts remain distinct so an interrupted operation does not become permission to adopt
unknown bytes. Each source family’s recovery method requires its own qualification and replay evidence.
The NPS catalog and downloadable manifest remain separate. Modern publisher cards and archival candidates have inde-
pendent ledgers; they are not synthesized into NPS records. A PDF enters an expanded analytical frame only after payload
integrity, source identity and extraction coverage support that admission. An unavailable request, contact-only card and
independently listed gap are different outcomes. The historical denominator remains unknown unless independent holdings
evidence establishes it.
Every published table or figure binds its consumed inputs, method, settings, runtime and artifacts. Readers recheck those
inputs before accepting a result. Immutable outputs preserve earlier generations and explicit failures. Source indexes retain
unresolved and unavailable records, while page search admits only the independently current physical-page scope.
2.3
Text units and recovery
A physical page is a zero-based position within an exact PDF digest. Its PDF page label and observed printed folio are
separate observations. Native text preserves source blocks and coordinates. Accepted OCR is a separately indexed derivative;
6

## Page 8

it is not fused into native counts. Empty recognition, normalized zero-token text, missing recognition and recognition failure
retain different meanings.
The newer language pass uses the recorded Unicode normalization and tokenizer. It preserves complete per-issue counts,
token opportunities, zero-token units and source-layer identities. The earlier alphabetic-token baseline remains available
under its own definition. Analyses never silently pool prose, references, code, tables, advertisements and captions into a
validated prose category.
Witnessed repository wrappers support a reversible exclusion view. A separate native-rich view uses the recorded extraction-
coverage rule and exact issue membership. These views change opportunities and sometimes the population under study.
They cannot repair missing historical material or establish that early scanned issues are represented by later native-rich issues.
OCR accuracy requires independent transcription, alignment rules and separate character, word, reading-order and
downstream-assignment assessments. Those accuracy estimates are not available here. A source-image comparison raised a
possible rotation-dependent omission; its cause must be tested on actual controlled raster PDFs before changing the frozen
recognition method. More recognized characters alone do not demonstrate correctness.
2.4
Article discovery and review
Discovery retains exact contents rows and a separate native-font view. The latter uses large folios and adjacent larger-than-
body title text, calibrated on an inspected 2022 contents page. It is a retrieval heuristic with unmeasured historical precision
and recall. All observed contents pages are excluded from the subsequent body-title search. A printed folio is never treated
as a PDF index. Repeated headings, cross-layer ambiguity and missing starts remain unresolved. Departments and notices
remain available for genre review.
The ledger distinguishes title observations, unique physical starts, layer starts, explicit article regions and source-checked
interpretations. An interval up to another title is a candidate reading window, not certified ownership of intervening pages.
Articles can share pages, continue nonconsecutively, or contain unnumbered insertions. Region order and overlap checks
establish internal consistency; they do not measure boundary accuracy.
Each executed reading records a question, contribution, setting, evidence basis, claims, limitations, implications and exact
supporting or contrary passages. Source locations retain their layer, PDF digest, physical page and occurrence. Implemen-
tation accounts, tutorials, conceptual arguments and empirical evaluations are appraised on their own evidence. Reported
improvements keep their units and study designs; incompatible before/after descriptions are not pooled into a causal effect.
The thematic 2022 comparison and the structured native-rich/early-OCR selections are development readings. Their specifica-
tions and selection failures remain visible. AI-assisted source checking is distinct from independent human review. Agreement
coeﬀicients, boundary accuracy and semantic-label accuracy remain unmeasured; methods for those tasks must match their
actual annotation units [Artstein and Poesio, 2008].
2.5
Language and statistical summaries
Frequency rates use complete eligible token opportunities. Issue prevalence and dispersion reveal whether a frequent term
reaches many issues or is concentrated in a few. Adjacent pairs restart at every page or explicit region; endpoint opportunities
accompany association scores. Diversity uses nonoverlapping 100-token windows, restarted at each unit, with short units
and discarded tails reported. Concordances retain complete target counts and bounded source-linked contexts.
The expanded exploratory contrast uses two issued-metadata cohorts: years at or before 1999 and at or after 2002. Intervening
and unparsed years are excluded.
The full two-by-two likelihood ratio ranks retrieval candidates; it supplies no token-
independence p value. Smoothed log-base-two rate ratios show direction and size. Whole-issue resampling preserves within-
issue dependence, while leaving between-issue temporal dependence, source coverage, extraction error and data-selected
vocabulary outside its nominal marginal intervals [Efron, 1979, Cameron and Miller, 2015]. The results report these sensitivity
views against their corresponding source frames. Complete descriptive counts do not acquire sampling uncertainty because
they are plotted.
Keyword screens and initialism expansions are retrieval instruments. They do not establish validated topics, meanings, author
intent or institutional adoption. Contexts and contrary examples precede interpretation [Grimmer and Stewart, 2013].
7

## Page 9

2.6
Visual measurements
Embedded raster assets, saved placements, native resource observations and rendered physical pages are different units. Base
rasters are measured without silently compositing unknown masks. Whole-page appearance uses the recorded rendering scale
and white RGB background, then measures pixel descriptors and spatial summaries on the full render. Policy refusals and
missing outputs remain outside the eligible denominator. A thumbnail is a viewing aid, not a new pixel measurement source.
Native layout is a separate derivative: span/font/size observations, character-weighted typography, column-start candidates
and the exact union of native rectangles clipped to the declared unrotated frame. No-native-rectangle coverage is an observed
zero for that measure; missing font size is unavailable. Neither condition establishes that the printed page is blank. Reported
font metadata and candidate reading order are not independently validated.
Source excerpts support bounded interpretation of visual form and argument. Raster entropy, nonwhite fractions and font
sizes cannot identify a photograph, an empirical chart, an architecture, an advertisement or a persuasive argument. Semantic
figure roles and figure–text relations need their own source-linked annotation and validation [Pfitzmann et al., 2022, Martinec
and Salway, 2005]. The cover groups the issue-front glyphs by year and outlines open print scopes. The text abstract shows
issue-normalized mixtures and page-weighted era totals. The page-shape abstract pools each aggregate darkness map into
a 12-by-17 grayscale grid and draws the cells as vectors.
To test whether open historical dates appeared on misfiled pages, we screened the opening and closing 8 admitted text lines
per retained page. The rule retrieves date-bearing lines at or near the journal masthead. We inspected each cross-date
candidate on the source-page raster and recorded what the marked date described. The screen measures detection under its
text and placement rules. Independent issue copies and actual printed leaves determine custody.
The folio searches use the retained text keyed by source PDF digest and physical page. For November 1999, the publisher’s
digital insertion and the neighboring printed pages provide known matches. We count nonoverlapping five-token sequences
specific to one query page and inspect the corresponding issued page whenever a different issue reaches eight such matches.
Clipped re-extraction tests which printed region accounts for the shared sequences. These comparisons assess retrieval on
the available text and do not estimate recall for unheld folios or poorly extracted pages.
2.6.1
Reconcile a figure across editions
For the November 1994 PASS error sequence, transcribe the event counts and denominators from the annual publisher reflow
and compare them to the corresponding Fairley teaching slide. Recompute the percentages separately for each denominator
and compare the pre-build, process and post-release counts to the NASA–contractor OI series, including the date of the last
observation. The resulting figure follows the arithmetic of one historical cohort. It does not estimate a general detection
rate across programs.
Reopen each retained PDF independently to recheck its digest and physical extent against the page ledger. Reconcile the
originally staged 53 appraisals by their full-reading index entries, page positions and receipt digests. This validates persistence
and evidence lineage for those appraisals.
2.6.2
Audit the boundaries of surviving bodies
For every physical position in the 11 retained PDFs with open original print scopes, compare the page image to the reading
register, qualified text and established continuations. Count an article body when its opening, ending and any intervening
text are present and the issue attribution is supported. Register recognizable fragments, abstract-only regions and disputed
attributions separately. Count embedded web copies within the held PDF and identify them separately from surviving print
bodies. Preserve links to prior complete appraisals to avoid counting the audit as new reviews.
The supplemental March 1992 comparison operates on four journal pages and twelve thesis appendix pages. Within each
journal page, a normalized OCR position counts once if it is included in an exact sequence-alignment block of eight or more
words to any one thesis page. Counted sequence windows and the longest exact block stay within a page. These statistics
depend on transcription and the comparison rule. The image-viewport screen measures native raster samples outside the
displayed rectangle and checks candidate decodes with independent readers. The screen separately records geometry, decoded
extent, and publication identity.
8

## Page 10

3
Results
3.1
Archive coverage and article evidence
The retained corpus comprises 281 issue PDFs and 9660 physical pages. Whole-page appearance measurements cover the
full retained set. Native extraction provides text on 7,102 pages, and optical extraction or transcription supplies searchable
text on a further 2,539. In total, 9641 physical pages are searchable (99.80%). The remaining 19 positions are verified blank
or scan-noise interleaves.
Exact article counts cover 270 of 281 retained issue identities (96.09%) under their attested source scopes. The retained
file supports the count for 258 identities and a matched publisher or retained alternate supports it for 12. The original
review queue has exact counts for 148 of 158 PDFs (93.67%). The 11 unresolved original print scopes include gaps where a
surviving preview, offprint, or secondary record establishes some content without establishing the original folio extent. The
total numbers of historically published issues and print articles remain unknown. Calhoun series
Direct complete article appraisals reach 8160 distinct physical pages (84.47%) through 2622 placements in 279 retained
issues. A further 16 printed excerpt or preview pages link to a complete web appraisal. The work register reconciles 2719
appearances and 11 repeated appearances into 2708 distinct complete AI-assisted appraisals. The apparent gap between
measured pages and direct article pages includes covers, administrative and ancillary pages, and material with other levels
of evidence. Independent human review of the whole archive remains open.
3.2
Language depends on what the archive makes readable
In the Calhoun native-text baseline, the smoothed log2 early-to-late token-rate ratios for security and agile were −3.83 and
−8.70, respectively. The catalog-issued cohorts use years through 1999 and from 2002 onward. The direction held in the
wrapper-excluded and native-rich views, while the eligible issue counts changed from 107 and 140 in raw native text to 24
and 134 in the native-rich view. The targeted OCR view also placed agile more often in the later cohort, with a log2 ratio
of −10.00 across 83 early and 138 late issues. The estimates describe different observed text populations. The figure’s 999
within-view issue resamples give marginal diagnostics conditional on those populations.
On the 279-PDF topic checkpoint, access to scanned prose changed both the amount of evidence and the apparent language
distribution. Among issues dated 1989–1999, adding historical OCR raised the issue-weighted delivery and process phrase
rate from 8.6% to 24.0% while the pooled page rate fell from 34.1% to 27.0%. Within the 82 issues available in both modes,
the mean paired increase was 20.0 percentage points. Trimming the outer three physical pages at each end changed the
early issue-weighted rate by just 0.1 percentage points among 104 matched issues. The two weighting schemes expose how
page opportunities and issue representation affect observed prominence.
The same checkpoint placed cybersecurity phrases on 22.8% of eligible pages in the average 2010–2019 issue. Intelligent-
systems phrases reached 17.3% in the average 2020–2025 issue across 11 PDFs and five observed years. Year-block reweight-
ing put the latter rate between 4.9% and 30.2% for the central 95% of resamples. Removing one year at a time gave 6.2%
to 19.9%. Dictionary sensitivity formed a separate check. Removing artificial intelligence yielded intelligent-systems rates of
3.5% in 1989–1999 and 14.1% in 2020–2025. Removing quality assurance reduced the overall assurance rate from 8.3% to
3.4%. The recent-frame interval is wide, and the assurance contrast shows how strongly some families depend on individual
phrases.
Independent implementations agreed on the phrase-screen output for 207 selected pages. Of 18 examined positive contexts,
eight were substantive, three bibliographic, three publication furniture, and four unclear. These bounded checks support
reproducibility of the counted pattern and expose ambiguity in its interpretation. The native-text contrasts similarly surfaced
plausible domain words alongside extraction fragments. The results characterize words in the observed text and leave the
prevalence of actual engineering practices to article-level evidence.
3.3
Page structure and publication form
The position of text within an issue mattered for some phrase families. Across 277 PDFs the mean within-issue difference
was +7.6 percentage points for delivery and process and +6.3 points for measurement. Central 95% whole-PDF resampling
ranges were +5.4 to +9.7 and +5.0 to +7.7 points, respectively. The corresponding cybersecurity difference was +0.3 points
with a range of −1.2 to +1.7. Page position therefore distinguishes some vocabulary families in this source frame, and its
value varies by family.
In 244 PDFs with a qualified front and 7,617 eligible later pages, the median front was 68.9 percentage points darker than
9

## Page 11

its issue’s median later page. After whole-page darkness was removed, the mean front-minus-interior center-to-ring contrast
was −4.4 points (central 95% whole-PDF resampling range −5.8 to −3.0). The direction held for magazine covers and
newsletter fronts when examined separately. A distinct issue-weighted grouping of all 9660 pages from 281 retained issues
achieved held-out adjusted Rand agreement of 0.990 to 1.000 across five refits. The groups capture reproducible spatial
patterns in the rendered pages. Those patterns contain contributions from layout, illustration, and scanning, so semantic
interpretation still requires page inspection.
3.4
What complete article readings add
The January 1998 publisher version of Don Estes’s working paper provides the conditions for a reversible treatment of two-
digit years and illustrates a proposed capture-and-replay test strategy. The independent arithmetic check covered 36159
calendar dates. This supports the transformation under the specified assumptions. The paper’s worked testing examples
leave field detection performance unmeasured. Original publisher paper
The April 2008 earned-value study examined when cost and schedule indices stabilized in project records from three settings.
Its stability criteria were met by 20% completion for three of 26 cost series and four of 37 schedule series. This empirical
example supplies observable thresholds and series denominators. Its inference is bounded by the observed projects and the
study’s definition of stability. April 2008 issue
Patti Spicer’s September–October 2010 article explains what the Common Criteria can establish about an evaluated product
and configuration, and where assurance about the surrounding installation still depends on deployment. Its examples illustrate
interpretation of a framework. The archived publisher article supplies the full argument and references, while independently
held issue copies preserve the announcement. The DTIC and Calhoun files align on 32 journal positions, including 31 with
equal normalized native text. Archived publisher article ⋅DTIC deposit
Joseph Arsenault’s June 1998 web article combines field experience with three descriptive inventories. Those tables document
reported uses and capabilities in the setting examined. They supply no matched performance comparison. The publisher
offprint has 3 pages and carries May footers, while the June directory and retained preview establish the web placement.
The offprint supplies a second presentation of the same work. Archived June article ⋅Publisher offprint
Glenn Booker’s August 2007 web column gives a control for expansion without a new article identity. The publisher version
contains 15 numbered observations, of which 11 match headings in the printed column. The 4 additional observations extend
a reflective argument. They supply an additional manifestation of the work, with no change to the distinct-work census.
Archived publisher column
3.5
What the publisher archived
The December 2017 back-issue directory and RSS feed contain 41 issue links whose periods match retained issues with exact
article counts. The December homepage identifies July–August as the latest issue. The archived directory and feed attest
4 issue releases in 2017. The two later planned issues have no verified release in the observed records. The cadence figure
marks that distinction. December homepage ⋅December issue feed
The 12 archived 2011–2012 issue listings contain 103 literal article links. 70 matched same-issue titles after normalization, 7
were variants checked against retained pages, and 24 used generic department labels. The other 2 were explicitly online-only.
These observations describe the captured indexes and their files. They leave both historical issue and article denominators
unknown.
The two most recent full-text additions extend the web record. A separate 8-page November–December 2010 publisher PDF
credits Wayne O’Brien and Raimund Merkert. Their paper illustrates design choices for integrating systems and supplies no
comparative performance result. Jeff Williams’s March–April 2011 web article spans 6 pages and proposes ways to make
software security decisions more visible and accountable. It reports no comparative outcome data. Both works support
appraisals of the authors’ reasoning. Neither resolves an original print scope. The January–February 2011 Design Point
source still supplies only an abstract and a publisher locator to an unverified full body. O’Brien and Merkert publisher PDF
⋅Williams publisher PDF ⋅Design Point issue listing
Independent citation evidence for an April 1996 software process improvement paper groups its Wiley and CrossTalk reports
under one 33-company study. The association limits independent-study credit. The original folios 26 and 27 remain unverified.
Fraunhofer review
10

## Page 12

3.5.1
Date signals and related publication copies
The text-edge screen inspected 9641 searchable physical positions and found 84 date signals. Visual inspection of the 3
cross-date candidates identified the known April 1991 leaf filed with May 1991. A reader letter in April–May 1992 referred
to March 1992, and a January–February 2012 reference listed a November 1999 conference. Neither citation changes the
printed-page inventory.
The ÉTS Montréal repository supplies Laporte and Trudel’s separate June 1999 conference paper. Raw PDF text extraction,
Unicode normalization and line-end dehyphenation identify 2594 exact contiguous five-token windows in common with the
already-reviewed November 1999 journal article on folios 8–13. The two documents describe the same investigation, and
the proceedings deposit leaves journal folio 22 and the center enclosure open.
A March 2002 archived distribution index lists 98 month-labeled links spanning January 1994 through March 2002, including
one combined November–December 1995 issue. This capture bounds the visible linked archive. The earlier publication
history remains open.
3.5.2
Follow the folio, then the text
In April 1996 the two retained folio controls ranked first in the issue search, which found no additional original leaf. The
March 1992 order notice names a later report form. A fresh optical review of all retained 1992 issues and a text search
across the wider corpus verified related examples with different issue dates or pagination and left the March enclosure open.
The November 1999 digital issue places 15 web pages between print folios 21 and 23. The December annual index identifies
the two web articles. All 17 known pages matched their retained equivalents among 9641 searchable positions. Two cross-
issue matches reached the retrieval threshold. January 2000 folio 19 shares the article heading and references, accounting
for 101 of 105 shared windows. January 1999 folio 23 repeats an author biography, accounting for 48 of 49 shared windows.
The printed November folio 22 and subscription form remain unlocated.
The KUTV archive catalogs tape 19900424 for local news on the April 1990 conference opening date. Its item description
gives no segment-level evidence of conference coverage. The June/July 1992 CrossTalk issue names the May 1991 program
contact and describes the QUES industry agreement. It corroborates the program and contact. The contemporaneous
reuse search checked 800 pages in 72 ACM RISKS Digest issues and 15 pages from two AWS Observer issues. No verified
CrossTalk anchors matched. These archival checks added zero original folios, exact article counts or complete appraisals.
3.5.3
One error cohort survives in a second carrier
Figure 1 in Peter Koester’s November 1994 article follows 123 Primary Avionics Software System errors. Inspection detected
92, pre-build work detected 22, and independent testing detected 7, leaving 2. The cumulative detections were 92.7% after
pre-build and 98.4% after all three stages. The annual publisher reflow and Fairley’s later teaching slide report the same
counts. The slide attributes detection of the last two to independent verification and validation. The earlier figure lists them
as remaining from the three displayed stages.
The NASA and contractor retrospective gives OI-23 the same 114 pre-build and 2 post-release count, alongside 8 process
reports against the journal sequence’s 7 detections in its independent-testing stage. NASA reports 124 total through STS-
135. The two accounts therefore differ in the middle count and use distinct category definitions and reporting windows. The
relationship between the cohorts remains unresolved.
3.5.4
Catalogs narrow the search for paper
The Navy REVIC distribution preserves a catalog entry for the 324-page January 1990 user-group proceedings and a 21-page
collection of 1990 newsletters. It lists the charter, bylaws and group history dated 8 March 1990. These are contemporaneous
publication leads for the retained March 1990 estimation tutorial.
The missing folio remains open.
The 1994 TABES
proceedings identify May 1990 CrossTalk speaker David Norton as director of the Utah Research Institute and give a Weber
State University mail code. This is an institutional search lead. For April 1996, the copyright register records a separate
23-page 1995 report by the article’s authors. The original April 1996 folios remain open.
3.5.5
What survives in the eleven open issues
The 11 PDFs with unresolved original scopes contain 197 physical pages and 66 confirmed complete bodies linked to existing
AI-assisted appraisals. The complete-body count includes 3 embedded web bodies. In the final five PDFs adjudicated, the
11

## Page 13

audit registered 11 incomplete bodies separately. A complete CLIPS/Ada notice on a page with an April footer is excluded
while its May 1991 attribution remains disputed. The November 1994 two-page retained file in this audit has zero confirmed
complete bodies. Its overlapping web or reflow content remains evidence at its own source grain.
The companion census gives per-file counts and links each audited file to the retained inventory by issue identifier and PDF
digest. Original print extents and article totals for these scopes remain unresolved.
3.5.6
Where prose and evidence diverge
Three publisher manifestations of the April 1996 survey agree on 22 rows in Table 4 and 5 in Table 5. The prose gives Level
3 defect profiling as 71%. Table 4 gives 75% at Level 3 and places the prose value at Level 2. The prose calls Level 3 process
compliance the highest at 50%. The table gives 100% at Level 4. Cell denominators and original print folios 26–27 remain
unverified.
For May 1991, literal phrase and author probes covered 1,602 extractable pages across four conference proceedings volumes.
They confirmed the 1989 Zultner antecedent and found no matching continuation. The remaining original page gaps stay
open.
The November 1994 usability case matches 19 words of its cited interface design handbook after normalizing punctuation.
Heiner reported 19 locations and forecast at least 30 by the end of 1995. The article describes site visits, joint design and
iteration. It reports no independently measured effect size. The Oﬀicial Congressional Directory locates the program’s parent
center, leaving custody of the original journal unverified.
3.5.7
Tracing the priority census
We rehashed the 158 retained PDFs on the original priority list. The 148 scopes with exact counts contain 1523 article
placements. The audit linked 1497 directly read placements and 26 alternate placements to specific complete appraisals.
The alternate placements include 7 February 1990 entries tied to the same works in the January copy. The audit rebound
all 53 previously staged appraisals to the admitted trace. 10 priority scopes still lack an exact original count. The historical
journal totals remain unknown.
3.5.8
What an original copy must establish
The retrieval docket distinguishes 11 retained PDFs with unresolved print extents from 4 early issues known through separate
witnesses. The early targets are July 1988 and July, October, and December 1989. The December 1989 thesis citation
identifies an article and its opening page. A Library of Virginia finding aid identifies accession 33720, box 1, as a potential
lead for the July 1989 obituary. The folder description does not name an item. A complete face with its printed borders,
sequence, and evidence of any insert placement can resolve an original-scope gap.
A refresh of 11 Calhoun catalogue records found unchanged Signposting linksets and the same advertised PDFs. Those
public relationships concern deposited copies. The catalogue does not identify any separate scan master or original paper
copy. The SofTech files and the National Archives accessions provide search locations without establishing custody of a
specific missing page.
3.5.9
Auditing continuity in source text and images
The August–September 1991 retained issue and a publisher reflow agree on 11 contents entries. Matching passages and
signed endings support 7 substantive identities as a lower bound. The July 1991 issue skipped a printed heading number
and left a visible item out of its contents, which limits the inference from an apparently continuous index. For the May 1991
software test environment work, the Woelfer thesis independently documents the participating organizations. Its normalized
text and the surviving Bollinger abstract share a maximum run of 5 words, leaving the full Bollinger paper unverified.
The raster screen considered 2,256 image placements across the 197 retained pages of the open scopes. 4 placements
extended beyond their page viewport and contained 137,767 native pixels outside it. These pixels extend held graphics and a
chart. Poppler and MuPDF agreed on the July JPEG payloads and all 8,415,000 decoded grayscale samples of the selected
March 1992 image. The July scans’ cut edges and the March scan’s white region are already in the held image data. The
public figures contain derived counts and aggregate imagery. The native raster audit supplies no original-folio credit.
12

## Page 14

3.5.10
How repeated observations change the evidence
An SEI case report and Brodman’s later account trace two April 1996 repair-cost values to the same Raytheon project.
Design-defect repair shifts from 0.75% to 2% of project cost and coding-defect repair from 2.5% to 4%. The two accounts
contribute one cohort. The November 1994 PASS comparison similarly gives one rounded inspection share of 92.7% in
both the multi-year IBM figure and the staged single-release count. Stage definitions and populations differ and preclude a
replication claim. A subsequent Inspector General audit gives 997 nonconforming records (10.80%) out of 9,229 approved
entries. DISA gives 881 (9.55%) on the same denominator after accounting for overlapping flags. The difference describes
two readings of one inventory.
3.5.11
A reengineering assessment across two publications
Charles A. Jennings’s Naval Postgraduate School thesis attributes Appendix A, printed pages 84–95, to the March 1992
Re-engineering Tools Report. The March CrossTalk article includes the same weighted questions, cost assumptions, and
three matrices. Figure labels and page layouts differ across copies.
The comparison matches 2,254 of 2,993 normalized journal OCR word positions in exact page-bounded blocks of eight or
more words (75.31%). The texts share 424 distinct 50-word sequences. The longest block runs 119 words on journal folio
8 and thesis page 88. The alignment figure (fig. 10) displays these diagnostics. They establish substantial reuse within this
comparison and corroborate an existing appraisal. The original order card, obscured material on folio 1, and historical print
counts remain unresolved.
4
What the surviving record can support
The surviving journal combines records of technical practice with advocacy about how that practice should be organized. Its
authors can document a design choice, a requirement or an institutional preference with considerable specificity. A design
diagram can specify where a control runs while leaving its effect unmeasured. A practitioner narrative can show how an
evaluation was conceived without providing a comparison group, baseline denominator or replication. Keeping the evidence
design in the article register makes these distinctions available for synthesis.
The strongest source comparisons hold several factors fixed at once. The same PDF hash fixes the bytes and page ordering. A
matched offprint can preserve the body while changing folios or an editorial wrapper. Matched text and aligned rendered pages
offer separate controls on visible content. The Spicer and Arsenault examples show how publisher-defined web placement
resolves relationships that the native document alone cannot. They also show why a recovered online article does not fill an
absent print folio.
Several historical boundaries remain. An apparent gap can be an unissued period, a known issue whose copy is lost, a
surviving excerpt, or a record attributed to the wrong period. Early scanning geometry can conceal text the printed copy
retained. Text-readable pages may omit a diagram, and visually measured pages may contain no recoverable writing. A
denominator for published issues or original articles requires contemporaneous numbered witnesses, complete independent
holdings or another source that actually bears on the missing material.
4.1
Limitations and future verification
The complete text checks are AI-assisted and source anchored. Independent human dual review, inter-rater agreement, OCR
error rates and validated automatic article segmentation are unresolved. The linguistic strata reflect source and editorial
history together. The nominal issue-level intervals capture sensitivity within the declared comparison, with no probability
model for archival absence or cross-period causal change. The page-shape clusters are geometric descriptions. Their stability
under refitting does not validate titles such as advertisement, architecture or evidence. No journal-wide causal effectiveness
estimate follows from the reviewed case reports.
4.2
Research artifacts and access
The Calhoun collection and the CrossTalk journal hold the original materials under their respective terms. Cite the original
articles for their authors’ claims. The versioned Zenodo deposit contains the preprint, bibliographic locators, checksums and
derived measurements. The matching GitHub source and build kit reproduce this edition.
13

## Page 15

5
Figures
5.1
Native sensitivity and historical OCR contrasts
CROSSTALK / ANALYTIC PLATE 05
Same direction, different denominator
FIXED EXAMPLES: SECURITY AND AGILE  /  NPS CATALOG-ISSUED COHORTS
VIEW / POSITIVE-TOKEN ISSUE COUNTS (EARLIER / LATER)
Earlier (<=1999) / later (>=2002) smoothed token-rate ratio
log2 scale
-14
-12
-10
-8
-6
-4
-2
0
security
Raw native
107 / 140
-3.83
Wrappers removed
59 / 134
-3.82
Native-rich
24 / 134
-3.81
agile
Raw native
107 / 140
-8.70
Wrappers removed
59 / 134
-8.69
Native-rich
24 / 134
-8.69
Targeted OCR
83 / 138
-10.00
Each line: marginal 95% diagnostic from 999 whole-issue resamples within that view.
EXPLORATORY  |  COHORT MEMBERSHIP VARIES  |  SOURCE SELECTION AND OCR ERROR UNMODELED
Figure 1. Shown fixed examples use smoothed log2 ratios of catalog-issued early to late token rates. Lines are marginal 95% diagnostics
from 999 whole-issue resamples within each distinct view. No across-view difference or historical cause is inferred.
14

## Page 16

5.2
The archive changes the topic map
CROSSTALK / ANALYTIC PLATE 17
The archive changes the topic map
EXACT PHRASES  /  ONE VOTE PER PDF  /  EXTRACTION SENSITIVITY
SHARE OF ELIGIBLE PAGES WITH A FAMILY MATCH
1989-1999
106 PDFs / 3,045 pages
2000-2009
115 PDFs / 3,819 pages
2010-2019
46 PDFs / 1,888 pages
2020-2025
11 PDFs / 743 pages
Acquisition
9.7%
11.6%
5.4%
2.9%
Architecture
7.9%
7.7%
7.3%
4.5%
Assurance
7.5%
9.9%
7.4%
4.3%
Cybersecurity
0.2%
9.6%
22.8%
18.7%
Delivery and process
24.0%
30.8%
20.1%
22.2%
Embedded and real time
5.8%
6.4%
4.7%
1.8%
Intelligent systems
4.9%
1.2%
1.6%
17.3%
Measurement
9.2%
13.4%
9.0%
3.6%
People and training
1.8%
1.0%
3.9%
1.6%
Darker cells indicate higher rates on a common 0% to 36% scale.
Families can overlap on the same page.
1989-1999 / DELIVERY AND PROCESS
The weighting reverses the OCR comparison
Equal issue
8.6%
24.0%
Pooled page
34.1%
27.0%
NATIVE
PLUS OCR
2020-2025 / INTELLIGENT SYSTEMS
11 retained PDFs across 5 represented years
0%
40%
Year-block replay range 4.9% to 30.2%
Text eligibility and issue weights change the question the percentage answers.
279 PDFS  |  9,547 PAGES TRACKED  |  LITERAL RETRIEVAL
Figure 2. Matching-page fractions give each PDF equal weight. The early delivery and process example compares native and admitted
text under two weightings. The recent intelligent-systems interval shows the effect of resampling represented years.
15

## Page 17

5.3
Which words carry the topic map?
CROSSTALK / ANALYTIC PLATE 19
Which words carry the topic map?
EXACT PHRASES  /  EQUAL PDF WEIGHT  /  ONE-PHRASE REMOVAL
FAMILY
MOST INFLUENTIAL PHRASE
FAMILY RATE LOST
Assurance
quality assurance
59%
People and training
training program
36%
Acquisition
software acquisition
33%
Architecture
software reuse
31%
Intelligent systems
artificial intelligence
28%
Measurement
earned value
26%
Cybersecurity
information assurance
26%
Embedded and real time
embedded systems
24%
Delivery and process
process improvement
22%
A page disappears only when no other phrase in that family still matches.
DELIVERY AND PROCESS
Largest unique contributor changes
2000-2009
process improvement
2020-2025
devsecops
INTELLIGENT SYSTEMS
Without 'artificial intelligence'
1989-1999
4.9%
3.5%
2020-2025
17.3%
14.1%
279 PDFS  |  FIXED PAGE DENOMINATORS  |  LITERAL RETRIEVAL SENSITIVITY
Figure 3. Each bar shows the fraction of a family rate lost after removing its most influential phrase. The examples show a change in
the period-specific leading process phrase and the intelligent-systems rates with and without artificial intelligence.
16

## Page 18

5.4
Do front-page patterns track the words inside?
CROSSTALK / MATCHED ISSUE ANALYSIS
Does the front track the words inside?
239 PDFs matched by exact bytes. The two estimates use the same issues.
Interior phrase rate and front geometry
Pearson correlation, one observation per retained PDF
-0.15
-0.10
-0.05
0
+0.05
+0.10
+0.15
Acquisition
Architecture
Assurance
Cybersecurity
Delivery and process
Embedded and real time
Intelligent systems
Measurement
People and training
Pooled
Within decade and front role
Where each front puts its ink
Equal-PDF mean centered front minus later-page median in each cell
1990s cover
24 PDFs
1990s newsletter
59 PDFs
2000s cover
115 PDFs
2010s cover
32 PDFs
2020s cover
9 PDFs
Less dark
More dark
White is zero. Full scale is 30 percentage points in either direction.
Figure 4. The paired symbols compare pooled and centered within-decade and front-role correlations on the same PDFs. The spatial
panels show equal-PDF mean per-cell contrasts relative to later pages in each PDF. Red indicates greater darkness and gray indicates
less.
17

## Page 19

5.5
Where full reading reaches
CROSSTALK / REVIEW COVERAGE
Where full reading reaches
A page is reached if a completely appraised article occupies any region on it.
Share of native interior topic-hit pages reached
Gray marks the prior edition. Red marks this edition.
0%
25%
50%
75%
100%
Acquisition
27%
Architecture
44%
Assurance
38%
Cybersecurity
68%
Delivery and process
33%
Embedded and real time
36%
Intelligent systems
71%
Measurement
33%
People and training
66%
Relative ink inside the same PDF
30 Calhoun PDFs with both page groups
-7.96 points in mean nonwhite area
Mean of the reached-minus-other differences within each PDF.
Red cells show more nonwhite area on reached pages.
Gray cells show less. White represents zero.
Shared cell scale: −13.9 to +13.9 percentage points.
Figure 5. Gray and red points show the fraction in editions 0.20 and 0.21 of native interior pages reached within each literal topic
family. The bottom grid compares local nonwhite area on reached and other interior pages from the same strict Calhoun PDF, then
gives each eligible PDF equal weight.
18

## Page 20

5.6
A preview, its publisher links, and the full article
THE PUBLISHER'S WEB RECORD
Two routes from a printed notice to a verified complete body
PRINT WITNESS
PUBLISHER ROUTE
VERIFIED FORM
Joseph Arsenault  /  June 1998
Preview in June issue
Opening matches the full HTML
June issue directory
Original HTML plus migrated PDF
HTML and 3-page offprint
PDF footers say May 1998
HTML: 2 original GIF ornaments
Patti Spicer  /  September·October 2010
Publisher's note
Names an online bonus article
Archived issue directory
Literal link to the original HTML
Complete article text
Two numbered HTML tables
0 linked figure binaries recovered
The two article sources add no original print folios. Extent is recorded in each source's own units.
Figure 6. Two CrossTalk works recovered through links in the oﬀicial archive. Arsenault’s typeset pagination and the publisher’s online
placement record different parts of its history. Spicer’s title announcement leads to a full HTML article. The original-format units differ,
so the paths trace evidence without a shared extent scale.
19

## Page 21

5.7
The publisher directory and feed together index the surviving run
The later publisher record
41 publisher links match retained issue scopes
Read each row from January to December. One cell represents a paired issue.
Jan / Feb
Mar / Apr
May / Jun
Jul / Aug
Sep / Oct
Nov / Dec
2010
2011
W
2012
2013
2014
2015
2016
2017
Black  |  archive directory    Gray  |  issue feed    Red outline  |  planned, release unverified
W  |  Jeff Williams's March-April 2011 feature survives as a separate publisher PDF.
The December 2017 publisher homepage still identifies July-August as the current issue.
Figure 7. Each filled square identifies a publisher link matched to a retained issue with an exact article scope. The July–August 2017
issue is linked in the feed. The final two outlines mark planned periods whose publication is unverified.
20

## Page 22

5.8
How the PASS error count fell
PRIMARY AVIONICS SOFTWARE SYSTEM
Errors remaining after each stage
November 1994  /  Peter Koester, Figure 1
Initial cohort
123
After inspection
31
-92
After pre-build work
9
-22
After independent testing
2
-7
123 errors in one reported cohort. 121 detected in three stages. 2 remained.
Figure 8. Koester’s Figure 1 as transcribed from the November 1994 publisher reflow and checked against Fairley’s teaching slide. All
bars share the same zero-to-123 scale. The separate NASA table uses different categories and reports 124 for OI-23.
21

## Page 23

5.9
Pages and articles in eleven open issues
CROSSTALK / OPEN ORIGINAL PRINT SCOPES
Pages and articles in eleven open issues
Each row describes retained material. Original print article counts remain unknown.
Issue
Retained physical pages
Confirmed complete bodies
Partial / disputed
0
10
20
30
40
50
0
5
10
15
1990-03
8
13
-  /  -
1990-04
6
3
-  /  -
1991-05
9
1
2  /  1
1991-06
11
8
-  /  -
1991-07
16
1
3  /  0
1991-08/09
9
3
4  /  0
1992-03
25
8
1  /  0
1994-11
2
0
-  /  -
1996-04
31
7
1  /  0
1999-11
47
11
-  /  -
2000-05
33
11
-  /  -
Complete retained body
Embedded web body
A dash in the last column means no value was recorded. Zero is an observed count.
Figure 9. The eleven open print scopes preserve 197 physical PDF pages and 66 confirmed complete bodies, including three embedded
web bodies. The last column records partial bodies and disputed regions where enumerated.
22

## Page 24

5.10
Exact wording across two 1992 versions
Exact wording across two 1992 versions
March CrossTalk article and Charles A. Jennings's reengineering appendix
Journal OCR word positions
2,993 total
75.31%
2,254 in exact blocks of at least 8 words
739 outside qualifying blocks
Longest exact block
119 words
Journal folio 8
Thesis page 88
Longest block observed by the
page-bounded alignment.
Shared distinct word sequences
0
1,000
2,000
4 words
1,969
5 words
1,940
6 words
1,897
8 words
1,788
12 words
1,579
25 words
1,043
50 words
424
Sources: CrossTalk, printed folios 7-10. Jennings thesis, printed pp. 84-95.
Figure 10. Alignment of the four March 1992 journal pages against twelve thesis appendix pages. The bar counts the union of OCR
word positions in page-bounded exact blocks at least eight words long. The sequence profile counts distinct contiguous word windows
observed in both sources. These are lexical measures under the stated transcription and matching rule.
23

## Page 25

6
Appendix. Source and evidence units
The issue inventory binds the observed source URL, physical extent and PDF hash for each retained issue. Issue/front
provenance uses publication order and a separate physical page identity. The page-measure table links an issue, digest and
zero-based physical index to the admitted text layer, reading role, page-shape assignment and mean darkness.
Evidence observation
Unit
Retained count
Acquisition and physical
appearance
Physical PDF pages
9660
Searchable native or optical
extraction
Physical PDF pages
9641
Complete direct readings
Physical PDF pages with reading
reach
8160
Original or matched-alternate
census
Attested retained issue scopes
270
Complete appraisals after linking
repeated manifestations
Distinct journal works
2708
Still unresolved
Original issue scopes
11
The companion tables supply the complete retained-source inventory and page rows as separate files. The publisher-cadence
table supplies the attested issue periods and their links to retained source digests. The inventory attaches a census disposition
to each retained source. The open original scope count also includes one issue whose exact retained-scan count does not
certify its original printed extent.
Cite this edition with the version DOI, 10.5281/zenodo.24297224. The concept DOI identifies the study across editions.
The companion data include the outcomes for the three print-date flags with issue digests and page positions.
The companion historical summary records the reviewed folio and archive checks, source report digests, and coverage credits.
24

## Page 26

References
Crosstalk: The defense software engineering journal. URL https://crosstalk.nps.edu/.
Crosstalk series in calhoun. URL https://calhoun.nps.edu/entities/series/fcb7e5ae-3429-46a2-b3b0-c7865552f1fe. Repository series
identity; catalog response completeness is distinct from historical issue completeness.
Ron Artstein and Massimo Poesio. Survey article: Inter-coder agreement for computational linguistics. Computational Linguistics, 34
(4):555–596, 2008. doi: 10.1162/coli.07-034-R2. URL https://aclanthology.org/J08-4004/.
Emily M. Bender and Batya Friedman. Data statements for natural language processing: Toward mitigating system bias and enabling
better science. Transactions of the Association for Computational Linguistics, 6:587–604, 2018. doi: 10.1162/tacl_a_00041. URL
https://aclanthology.org/Q18-1041/.
Frederick P. Brooks, Jr. No silver bullet: Essence and accidents of software engineering. Computer, 20(4):10–19, 1987. doi: 10.1109/
MC.1987.1663532.
A. Colin Cameron and Douglas L. Miller. A practitioner’s guide to cluster-robust inference. Journal of Human Resources, 50(2):317–372,
2015. doi: 10.3368/jhr.50.2.317.
Defense Science Board. Report of the defense science board task force on acquiring defense software commercially. Technical Report
ADA286411, Oﬀice of the Under Secretary of Defense for Acquisition and Technology, June 1994. URL https://stib.cto.mil/wp-
content/uploads/2025/10/94-9_Acquiring-Defense-Software-Commercially.pdf.
Bradley Efron. Bootstrap methods: Another look at the jackknife. The Annals of Statistics, 7(1):1–26, 1979. doi: 10.1214/aos/1176
344552.
Justin Grimmer and Brandon M. Stewart. Text as data: The promise and pitfalls of automatic content analysis methods for political
texts. Political Analysis, 21(3):267–297, 2013. doi: 10.1093/pan/mps028. URL https://doi.org/10.1093/pan/mps028.
Radan Martinec and Andrew Salway. A system for image–text relations in new (and old) media. Visual Communication, 4(3):337–371,
2005. doi: 10.1177/1470357205055928. URL https://journals.sagepub.com/doi/10.1177/1470357205055928.
David Moher, Larissa Shamseer, Mike Clarke, Davina Ghersi, Alessandro Liberati, Mark Petticrew, Paul Shekelle, Lesley A. Stewart,
and PRISMA-P Group. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement.
Systematic Reviews, 4:1, 2015. doi: 10.1186/2046-4053-4-1. URL https://link.springer.com/article/10.1186/2046-4053-4-1.
Peter Naur and Brian Randell, editors. Software Engineering: Report on a Conference Sponsored by the NATO Science Committee.
NATO Scientific Affairs Division, 1969. URL https://homepages.cs.ncl.ac.uk/brian.randell/NATO/nato1968.PDF. Conference held
in Garmisch, Germany, 7–11 October 1968.
Wanda J. Orlikowski and JoAnne Yates. Genre repertoire: The structuring of communicative practices in organizations. Administrative
Science Quarterly, 39(4):541–574, 1994. doi: 10.2307/2393771. URL https://ccs.mit.edu/papers/ccswp166.html.
Matthew J. Page, Joanne E. McKenzie, Patrick M. Bossuyt, Isabelle Boutron, Tammy C. Hoffmann, Cynthia D. Mulrow, et al. The
PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ, 372:n71, 2021. doi: 10.1136/bmj.n71. URL
https://pmc.ncbi.nlm.nih.gov/articles/8005924/.
Gary A. Petersen. Crosstalk: The long and winding road. CrossTalk, page 13, August 2008. URL https://calhoun.nps.edu/entities/p
ublication/6965921b-6767-4425-9af3-7a39eeb0f492. Opening page inspected; retrospective historical testimony.
Kai Petersen, Sairam Vakkalanka, and Ludwik Kuzniarz. Guidelines for conducting systematic mapping studies in software engineering:
An update. Information and Software Technology, 64:1–18, 2015. doi: 10.1016/j.infsof.2015.03.007. URL https://www.sciencedir
ect.com/science/article/pii/S0950584915000646.
Birgit Pfitzmann, Christoph Auer, Michele Dolfi, Ahmed S. Nassar, and Peter Staar. DocLayNet: A large human-annotated dataset
for document-layout segmentation. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining,
pages 3743–3751, 2022. doi: 10.1145/3534678.3539043. URL https://doi.org/10.1145/3534678.3539043.
Hironori Washizaki, editor. Guide to the Software Engineering Body of Knowledge. IEEE Computer Society, 4 edition, 2024. URL
https://www.computer.org/education/bodies-of-knowledge/software-engineering.
25


---
*Extraction method: pymupdf*
