News Text Analytics: From Dataset Patterns to Responsible Decisions

Gareth Grant | Portfolio and learning notebook

A step-by-step investigation of 44,898 supplied news records. We compare simple classifiers, investigate shortcut learning, and explore topics. Each code cell has one learning objective and a space for comments.

Question: How well can article language distinguish the labels in these two files, and what limits its usefulness outside this dataset?

The labels are inherited from Fake.csv (0) and True.csv (1). They are not independently verified in this project. A model score is not a probability that a claim is true.

Route: setup → quality audit → development/test split → exploration → baseline → model comparison → independent test → explanations → topics → practical applications.

Run: Python 3.11+ with pandas, numpy, scipy, scikit-learn, matplotlib and IPython. Put both CSV files beside this notebook (or in data/), change DATA_DIR below, then run cells in order. No NLTK downloads, Colab uploads, or installation commands are needed. The published outputs were executed on the supplied files; rerunning replaces them. Full training may take several minutes.

Publication: Do not redistribute article text without checking rights and dataset terms. The HTML companion suppresses raw article excerpts. Keep full datasets outside your public website repository.

1. Import tools

Each tool has a clear purpose: tables, numeric arrays, reproducible models, metrics and charts.

Your comment: Add your observation or question here.

Code cell 1

from pathlib import Path
import re, json, platform, hashlib
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import sklearn
from IPython.display import display, Markdown
from sklearn.model_selection import train_test_split
from sklearn.pipeline import Pipeline
from sklearn.feature_extraction.text import TfidfVectorizer, CountVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.naive_bayes import MultinomialNB
from sklearn.dummy import DummyClassifier
from sklearn.decomposition import LatentDirichletAllocation
from sklearn.metrics import accuracy_score, balanced_accuracy_score, f1_score, precision_score, recall_score, roc_auc_score, classification_report, ConfusionMatrixDisplay, RocCurveDisplay, brier_score_loss
from sklearn.calibration import CalibrationDisplay

2. Configure paths and reproducibility

This is the only path you should normally need to edit. The random seed makes splits repeatable.

Your comment: Add your observation or question here.

Code cell 2

DATA_DIR = Path('.')  # Change to Path('data') if needed.
SEED = 42
LABELS = {0: 'Fake-file label', 1: 'True-file label'}
plt.rcParams.update({'figure.figsize': (8, 4), 'axes.spines.top': False, 'axes.spines.right': False})

3. Record the environment

Versions matter when reproducing numeric results.

Your comment: Add your observation or question here.

Code cell 3

display(pd.Series({'Python': platform.python_version(), 'pandas': pd.__version__, 'numpy': np.__version__, 'scikit-learn': sklearn.__version__}, name='version').to_frame())

	version
Python	3.12.14
pandas	2.2.3
numpy	2.3.5
scikit-learn	1.8.0

4. Load the fake-labelled file

Read one file at a time so failures are easy to locate.

Your comment: Add your observation or question here.

Code cell 4

fake = pd.read_csv(DATA_DIR / 'Fake.csv')
print('Fake rows:', len(fake))

Fake rows: 23481

5. Load the true-labelled file

The labels describe the source files, not the outcome of our own fact-check.

Your comment: Add your observation or question here.

Code cell 5

true = pd.read_csv(DATA_DIR / 'True.csv')
print('True rows:', len(true))

True rows: 21417

6. Validate the schema

Fail early if a required column is missing.

Your comment: Add your observation or question here.

Code cell 6

required = {'title', 'text', 'subject', 'date'}
assert required <= set(fake.columns) and required <= set(true.columns)
print('Required columns found in both files.')

Required columns found in both files.

7. Attach labels and combine

Use a new table rather than modifying the imported originals.

Your comment: Add your observation or question here.

Code cell 7

raw = pd.concat([fake.assign(label=0), true.assign(label=1)], ignore_index=True)
print('Raw shape:', raw.shape)

Raw shape: (44898, 5)

8. Check missing values

Missing values and blank strings are different problems.

Your comment: Add your observation or question here.

Code cell 8

display(raw.isna().sum().rename('missing').to_frame())

	missing
title	0
text	0
subject	0
date	0
label	0

9. Build a conservative article key

Lowercase and whitespace normalization identify exact content copies with superficial formatting differences. This is not semantic deduplication.

Your comment: Add your observation or question here.

Code cell 9

data = raw.copy()
data['article'] = data['title'].fillna('') + ' ' + data['text'].fillna('')
data['key'] = data['article'].str.lower().str.replace(r'\s+', ' ', regex=True).str.strip()
print('Empty combined articles:', data['key'].eq('').sum())

Empty combined articles: 0

10. Audit contradictory labels

Identical text with both labels cannot provide a consistent training target.

Your comment: Add your observation or question here.

Code cell 10

conflict_keys = data.groupby('key')['label'].nunique()
conflict_keys = conflict_keys[conflict_keys > 1].index
print('Conflicting exact article keys:', len(conflict_keys))

Conflicting exact article keys: 0

11. Remove contradictions, blank bodies and exact copies

A nonempty headline can hide an empty body. We require a body for the article model.

Your comment: Add your observation or question here.

Code cell 11

data = data[~data['key'].isin(conflict_keys) & data['text'].fillna('').str.strip().ne('')].copy()
data = data.drop_duplicates('key')
print('Rows after content cleaning:', len(data))

Rows after content cleaning: 38653

12. Prevent repeated titles or bodies crossing splits

Keep one record per normalized nonempty title and per normalized body. This conservative choice may remove legitimate updates; document the loss. Near-duplicate stories can still remain.

Your comment: Add your observation or question here.

Code cell 12

data['title_key'] = data['title'].fillna('').str.lower().str.replace(r'\s+', ' ', regex=True).str.strip()
data['body_key'] = data['text'].fillna('').str.lower().str.replace(r'\s+', ' ', regex=True).str.strip()
data = data[data['title_key'].ne('')].drop_duplicates('title_key').drop_duplicates('body_key').reset_index(drop=True)
print('Final rows:', len(data), '| Total removed:', len(raw) - len(data))

Final rows: 38262 | Total removed: 6636

13. Inspect metadata shortcuts

Subject is excluded from predictors. Its class separation reveals a collection shortcut in the original design.

Your comment: Add your observation or question here.

Code cell 13

display(pd.crosstab(data['subject'], data['label']).rename(columns=LABELS))

label	Fake-file label	True-file label
subject
Government News	514	0
News	9049	0
US_News	783	0
left-news	681	0
politics	6417	0
politicsNews	0	11145
worldnews	0	9673

Interpretation: The subject categories do not overlap between the files. Including them can reveal the label directly. Excluding this column is necessary, but other source/style cues can remain inside the article body. Deduplication reduces repeated-document contamination; it cannot establish publisher independence.

14. Make the independent test split

Reserve 20% before exploring language or fitting any vocabulary. Stratification keeps label proportions similar.

Your comment: Add your observation or question here.

Code cell 14

development, test = train_test_split(data, test_size=0.20, stratify=data['label'], random_state=SEED)

15. Make the validation split

A further 20% of development data becomes validation: approximately 64/16/20 overall.

Your comment: Add your observation or question here.

Code cell 15

train, valid = train_test_split(development, test_size=0.20, stratify=development['label'], random_state=SEED)

16. Verify split isolation

These assertions check exact normalized content, title and body isolation.

Your comment: Add your observation or question here.

Code cell 16

for column in ['key', 'title_key', 'body_key']:
    assert set(train[column]).isdisjoint(valid[column])
    assert set(development[column]).isdisjoint(test[column])
display(pd.DataFrame({'train': train['label'].value_counts(), 'validation': valid['label'].value_counts(), 'test': test['label'].value_counts()}).rename(index=LABELS))

	train	validation	test
label
True-file label	13323	3331	4164
Fake-file label	11164	2791	3489

17. Plot training label balance

Use counts with explicit labels. The original pie chart risked mismatching value-count order and labels.

Your comment: Add your observation or question here.

Code cell 17

train['label'].map(LABELS).value_counts().plot.bar(color=['#16324F', '#FF2400'])
plt.ylabel('Articles'); plt.title('Training labels'); plt.xticks(rotation=0); plt.tight_layout(); plt.show()

18. Summarise training article length

Explore training only. Differences in length can be a shortcut rather than evidence of truth.

Your comment: Add your observation or question here.

Code cell 18

word_counts = train['article'].str.split().str.len()
display(word_counts.groupby(train['label'].map(LABELS)).describe().round(1))

	count	mean	std	min	25%	50%	75%	max
label
Fake-file label	11164.0	438.3	356.4	2.0	291.0	390.0	519.0	8148.0
True-file label	13323.0	393.0	271.7	33.0	160.0	370.0	527.0	5181.0

19. Measure a source marker

This measures the observed Reuters marker in your supplied training data, without assuming it proves reliability.

Your comment: Add your observation or question here.

Code cell 19

marker = train['article'].str.contains(r'\breuters\b', case=False, regex=True)
display(marker.groupby(train['label'].map(LABELS)).mean().mul(100).rename('Contains Reuters (%)').round(2).to_frame())

	Contains Reuters (%)
label
Fake-file label	1.19
True-file label	99.85

20. Define transparent text cleaning

Remove the obvious Reuters token and URLs, but retain negation, numbers and word order. More aggressive cleaning is not automatically better.

Your comment: Add your observation or question here.

Code cell 20

def clean_article(value):
    value = re.sub(r'https?://\S+|www\.\S+', ' ', str(value))
    value = re.sub(r'\breuters\b', ' ', value, flags=re.I)
    return re.sub(r'\s+', ' ', value).strip().lower()

21. Apply fixed cleaning

This function learns no dataset statistics. TF-IDF will still be fitted only on training data.

Your comment: Add your observation or question here.

Code cell 21

train_text = train['article'].map(clean_article)
valid_text = valid['article'].map(clean_article)

22. Fit a majority baseline

A baseline establishes the value added by language modelling. No oversampling is justified by the modest imbalance alone.

Your comment: Add your observation or question here.

Code cell 22

baseline = DummyClassifier(strategy='most_frequent')
baseline.fit(np.zeros((len(train), 1)), train['label'])
baseline_pred = baseline.predict(np.zeros((len(valid), 1)))
print('Validation baseline accuracy:', round(accuracy_score(valid['label'], baseline_pred), 4))

Validation baseline accuracy: 0.5441

23. Define the TF-IDF representation

TF-IDF weights terms using training frequencies. Bigrams capture short phrases; limits keep memory manageable.

Your comment: Add your observation or question here.

Code cell 23

def vectorizer():
    return TfidfVectorizer(ngram_range=(1, 2), min_df=3, max_df=0.95, max_features=30000, sublinear_tf=True)

24. Build logistic regression

The pipeline prevents the validation/test vocabulary from informing training. C=1 is fixed in advance.

Your comment: Add your observation or question here.

Code cell 24

logistic = Pipeline([('tfidf', vectorizer()), ('classifier', LogisticRegression(C=1.0, max_iter=1000, solver='liblinear', random_state=SEED))])

25. Train logistic regression

Fit the vectorizer and classifier together using training articles only.

Your comment: Add your observation or question here.

Code cell 25

logistic.fit(train_text, train['label'])
print('Logistic regression fitted.')

Logistic regression fitted.

26. Build and train Naive Bayes

This is a fast, meaningful text-classification comparator using the same representation.

Your comment: Add your observation or question here.

Code cell 26

naive_bayes = Pipeline([('tfidf', vectorizer()), ('classifier', MultinomialNB(alpha=1.0))])
naive_bayes.fit(train_text, train['label'])
print('Naive Bayes fitted.')

Naive Bayes fitted.

27. Define evaluation metrics

Macro-F1 gives equal weight to both labels. Fake-label recall uses positive label 0 explicitly.

Your comment: Add your observation or question here.

Code cell 27

def metrics(y, pred):
    return {'accuracy': accuracy_score(y, pred), 'balanced_accuracy': balanced_accuracy_score(y, pred), 'macro_f1': f1_score(y, pred, average='macro'), 'fake_precision': precision_score(y, pred, pos_label=0, zero_division=0), 'fake_recall': recall_score(y, pred, pos_label=0, zero_division=0)}

28. Compare validation results

Choose the best model on validation macro-F1, not on the independent test.

Your comment: Add your observation or question here.

Code cell 28

models = {'Logistic regression': logistic, 'Naive Bayes': naive_bayes}
validation_results = pd.DataFrame({name: metrics(valid['label'], model.predict(valid_text)) for name, model in models.items()}).T
validation_results.loc['Majority baseline'] = metrics(valid['label'], baseline_pred)
display(validation_results.round(4))

	accuracy	balanced_accuracy	macro_f1	fake_precision	fake_recall
Logistic regression	0.9868	0.9862	0.9867	0.9909	0.9799
Naive Bayes	0.9572	0.9566	0.9568	0.9557	0.9502
Majority baseline	0.5441	0.5000	0.3524	0.0000	0.0000

29. Select the model

Only trained text models are eligible; the baseline remains a reference.

Your comment: Add your observation or question here.

Code cell 29

best_name = validation_results.loc[list(models), 'macro_f1'].idxmax()
print('Selected on validation:', best_name)

Selected on validation: Logistic regression

30. Refit on all development articles

Reuse the fixed specification. The independent test remains unseen by every fitted component.

Your comment: Add your observation or question here.

Code cell 30

from sklearn.base import clone
final_model = clone(models[best_name])
final_model.fit(development['article'].map(clean_article), development['label'])
print('Final model fitted on', len(development), 'development articles.')

Final model fitted on 30609 development articles.

31. Evaluate the independent test

This is the primary performance estimate for this dataset and split.

Your comment: Add your observation or question here.

Code cell 31

test_text = test['article'].map(clean_article)
test_pred = final_model.predict(test_text)
test_prob = final_model.predict_proba(test_text)[:, list(final_model.classes_).index(1)]
test_metrics = metrics(test['label'], test_pred)
display(pd.Series(test_metrics).round(4).to_frame('test'))
print(classification_report(test['label'], test_pred, target_names=[LABELS[0], LABELS[1]], digits=4))

                 precision    recall  f1-score   support

Fake-file label     0.9942    0.9762    0.9851      3489
True-file label     0.9804    0.9952    0.9877      4164

       accuracy                         0.9865      7653
      macro avg     0.9873    0.9857    0.9864      7653
   weighted avg     0.9867    0.9865    0.9865      7653

	test
accuracy	0.9865
balanced_accuracy	0.9857
macro_f1	0.9864
fake_precision	0.9942
fake_recall	0.9762

32. Visualise classification errors

Rows are true labels; columns are model predictions. A true-file article flagged fake is a reputationally costly error.

Your comment: Add your observation or question here.

Code cell 32

ConfusionMatrixDisplay.from_predictions(test['label'], test_pred, display_labels=[LABELS[0], LABELS[1]], cmap='Blues')
plt.title('Independent test confusion matrix'); plt.tight_layout(); plt.show()

33. Assess probability ranking

ROC-AUC measures ranking across thresholds, not factual verification or probability calibration.

Your comment: Add your observation or question here.

Code cell 33

print('Test ROC-AUC:', round(roc_auc_score(test['label'], test_prob), 4))
RocCurveDisplay.from_predictions(test['label'], test_prob)
plt.title('Ranking the True-file label'); plt.tight_layout(); plt.show()

Test ROC-AUC: 0.999

34. Inspect probability calibration

Compare predicted True-file label scores with observed label frequency. Scores need external calibration before operational decisions.

Your comment: Add your observation or question here.

Code cell 34

print('Brier score (lower is better):', round(brier_score_loss(test['label'], test_prob), 4))
CalibrationDisplay.from_predictions(test['label'], test_prob, n_bins=10, strategy='quantile')
plt.title('Test calibration by label'); plt.tight_layout(); plt.show()

Brier score (lower is better): 0.0155

35. Estimate conditional uncertainty

Bootstrap test predictions, not the training pipeline. This interval omits retraining, source shift and correlated-story uncertainty.

Your comment: Add your observation or question here.

Code cell 35

rng = np.random.default_rng(SEED)
y_array = test['label'].to_numpy()
bootstrap_f1 = []
for _ in range(300):
    indices = rng.integers(0, len(test), len(test))
    bootstrap_f1.append(f1_score(y_array[indices], test_pred[indices], average='macro'))
print('Conditional 95% bootstrap interval for macro-F1:', np.quantile(bootstrap_f1, [0.025, 0.975]).round(4))

Conditional 95% bootstrap interval for macro-F1: [0.9835 0.989 ]

36. Train a headline-only stress test

Headlines reduce body dateline cues. This additional specification is diagnostic and does not replace the selected model.

Your comment: Add your observation or question here.

Code cell 36

headline_model = clone(logistic)
headline_model.fit(development['title'].map(clean_article), development['label'])
headline_pred = headline_model.predict(test['title'].map(clean_article))
display(pd.DataFrame({'Article model': test_metrics, 'Headline diagnostic': metrics(test['label'], headline_pred)}).T.round(4))

	accuracy	balanced_accuracy	macro_f1	fake_precision	fake_recall
Article model	0.9865	0.9857	0.9864	0.9942	0.9762
Headline diagnostic	0.9479	0.9460	0.9473	0.9593	0.9249

Interpretation: Compare the headline result with the article model. A performance drop indicates that body content helps, but does not identify whether that help comes from facts, topics, formatting or publisher style. Removing one source token does not remove every source cue. The test is a random within-dataset holdout; it is not an unseen-publisher or future-period evaluation. Do not keep revising the model against these test results.

37. Count errors without republishing articles

Error summaries are useful on a website and avoid printing article bodies.

Your comment: Add your observation or question here.

Code cell 37

errors = test.assign(predicted=test_pred, true_label_score=test_prob)
errors = errors[errors['label'] != errors['predicted']]
display(errors.groupby(['label', 'predicted']).size().rename('errors').to_frame())

		errors
label	predicted
0	1	83
1	0	20

38. Explain logistic regression word associations

Coefficients describe associations after other features are considered. They do not establish causation or deception.

Your comment: Add your observation or question here.

Code cell 38

terms = logistic.named_steps['tfidf'].get_feature_names_out()
weights = logistic.named_steps['classifier'].coef_[0]
term_table = pd.DataFrame({'term': terms, 'coefficient': weights})
display(pd.concat([term_table.nsmallest(10, 'coefficient').assign(direction='Fake-file association'), term_table.nlargest(10, 'coefficient').assign(direction='True-file association')], ignore_index=True))

	term	coefficient	direction
0	video	-8.775696	Fake-file association
1	via	-8.457162	Fake-file association
2	read more	-5.067111	Fake-file association
3	image	-4.725431	Fake-file association
4	just	-4.713479	Fake-file association
5	this	-4.575586	Fake-file association
6	read	-4.466633	Fake-file association
7	featured image	-4.434204	Fake-file association
8	featured	-4.385447	Fake-file association
9	you	-4.018811	Fake-file association
10	said	9.719024	True-file association
11	said on	6.921651	True-file association
12	washington	5.520089	True-file association
13	on wednesday	4.538372	True-file association
14	on tuesday	4.400306	True-file association
15	on	4.267287	True-file association
16	on thursday	4.223689	True-file association
17	president donald	4.153672	True-file association
18	said in	4.150258	True-file association
19	on monday	3.874062	True-file association

Explore topics separately

The original notebook learned LDA topics before its train/test split and then used those features for classification. Here LDA is a separate exploratory analysis trained only on a reproducible sample of training articles. Topic quality is not classifier accuracy. Sentiment and word clouds were removed from the core workflow: sentiment of stemmed text is difficult to interpret, and word clouds do not demonstrate predictive value.

39. Select a reproducible topic sample

A 3,000-article sample keeps this learning exercise practical. It is not a comprehensive map of all news.

Your comment: Add your observation or question here.

Code cell 39

topic_sample = train.sample(n=min(3000, len(train)), random_state=SEED)['article'].map(clean_article)

40. Create word counts for LDA

LDA uses counts. The English stop-word choice is specific to this exploratory topic task.

Your comment: Add your observation or question here.

Code cell 40

topic_vectorizer = CountVectorizer(stop_words='english', min_df=5, max_df=0.8, max_features=5000)
topic_counts = topic_vectorizer.fit_transform(topic_sample)
print('Topic count matrix:', topic_counts.shape)

Topic count matrix: (3000, 5000)

41. Fit a small topic model

Six topics and five passes are declared choices, not a claim of an optimal number of topics.

Your comment: Add your observation or question here.

Code cell 41

topic_model = LatentDirichletAllocation(n_components=6, max_iter=5, learning_method='online', random_state=SEED, n_jobs=1)
topic_model.fit(topic_counts)
print('Topic model fitted.')

Topic model fitted.

42. Inspect topic words

Use these words to suggest topic names, then verify by reading documents locally. A topic is not a true/fake label.

Your comment: Add your observation or question here.

Code cell 42

topic_terms = topic_vectorizer.get_feature_names_out()
topic_rows = [{'topic': i + 1, 'top_words': ', '.join(topic_terms[component.argsort()[-10:][::-1]])} for i, component in enumerate(topic_model.components_)]
display(pd.DataFrame(topic_rows))

	topic	top_words
0	1	police, said, people, city, black, media, gun,...
1	2	trump, president, clinton, said, donald, campa...
2	3	said, trump, north, united, china, russia, kor...
3	4	said, government, minister, party, state, pres...
4	5	said, house, state, tax, trump, senate, presid...
5	6	people, just, trump, video, twitter, like, new...

43. Plot average topic weights

These are average mixture weights within the sampled training corpus, not counts of unique stories.

Your comment: Add your observation or question here.

Code cell 43

topic_weights = topic_model.transform(topic_counts).mean(axis=0)
plt.bar(np.arange(1, 7), topic_weights, color='#16324F')
plt.xlabel('Topic'); plt.ylabel('Average mixture weight'); plt.title('Training-sample topic mixture'); plt.tight_layout(); plt.show()

44. Generate an evidence-based portfolio conclusion

This cell updates the wording when results change. Avoid copying an old percentage into a new run.

Your comment: Add your observation or question here.

Code cell 44

display(Markdown(f'**Measured result:** The selected {best_name} achieved test accuracy **{test_metrics["accuracy"]:.2%}** and macro-F1 **{test_metrics["macro_f1"]:.4f}** on **{len(test):,}** held-out articles after conservative deduplication. These results demonstrate recognition of labels in this dataset; they do not validate automatic fact-checking in new domains.'))

Measured result: The selected Logistic regression achieved test accuracy 98.65% and macro-F1 0.9864 on 7,653 held-out articles after conservative deduplication. These results demonstrate recognition of labels in this dataset; they do not validate automatic fact-checking in new domains.

What this shows a recruiter or client

This project demonstrates data-quality checks, reproducible splits, leakage prevention, sparse text features, baseline comparison, validation-based selection, uncertainty estimation, error analysis, explainability, topic discovery and responsible interpretation. Its strongest contribution is an auditable workflow rather than an inflated accuracy claim.

Business applications — proposals, not measured deployments

Decision	How this workflow could help	Data needed	Success measure
Brand and reputation monitoring	Rank incoming coverage for analyst review and group emerging themes	Current coverage, analyst annotations, sources and dates	Precision in the top review queue; analyst time per reviewed item
Competitive intelligence	Group product, competitor and customer themes	Licensed industry articles and sector taxonomy	Topic usefulness judged by analysts; missed important stories
Compliance and audit	Log ingestion, model version, score, review outcome and evidence	Authorised records plus verified review decisions	Traceability completeness and review error rates

A sensible pilot compares analysts with and without the tool on the same incoming stream. Estimate benefit as verified time saved × loaded hourly cost, minus running/review costs and false-alert handling. This notebook has not measured return on investment. Scores should prioritise review; factual decisions require evidence.

Astronomy — a transfer of method

I interpret “astrominor” as astronomy/astronomers. This news dataset contains no validated astronomy task, so the trained news classifier cannot be presented as an astronomy model.

A useful astronomy project would apply topic modelling to paper abstracts, tag papers by research area, or route observatory incident reports to specialists. NASA has documented LDA/NLP analysis of astrophysics research trends, and ADS supports astronomy literature discovery (sources below). Those examples motivate the transfer; they do not validate this notebook on astronomy data.

Build a new domain corpus with permissions, expert topic labels and paper dates. Hold out later papers and related paper families. Compare retrieval recall@k or expert-agreed tagging macro-F1 against a simple keyword baseline. Preserve negation, units and object identifiers. Literature topic trends can inform research planning; they cannot confirm astronomical discoveries.

Government — human-reviewed information triage

Possible pilots include grouping public consultation responses, routing service complaints, or prioritising potentially misleading public-health claims for qualified review. Each needs local, current, multilingual data and a task-specific label definition. This historical English news corpus provides no evidence of performance in South African public services.

Measure correct routing, urgent-case recall, reviewer workload and error rates by language and source. Keep evidence links, review logs, access controls, a correction route and retention rules. Do not use this model alone for censorship, penalties, eligibility or allegations about individuals. Its confidence is about dataset labels, not legal or factual truth.

Limitations and next experiments

1. Source and topic confounding remain even after removing subject and the Reuters token. Obtain publisher identifiers and use source-held-out evaluation.

2. Related stories may remain despite exact normalized deduplication. Cluster near-duplicates and keep each cluster in one split.

3. The date field is retained for future work, but inconsistent formats need auditing before a chronological split. Test on later independently labelled articles.

4. Labels were inherited; independently fact-check a stratified sample with multiple reviewers and report agreement.

5. Probabilities and uncertainty are conditional on this corpus. Evaluate calibration and threshold trade-offs on new development data before any operational pilot.

6. Topic number was fixed for readability. Assess topic stability across seeds and expert usefulness on training/development data.

7. The original SMOTE and TF-IDF objects were unused; the active classifier used topic distributions. The rebuilt notebook states exactly which models are actually trained.

Sources and further reading

- Supplied dataset’s stated Kaggle source: confirm licensing and provenance before publication.

- scikit-learn: common pitfalls and leakage: why preprocessing belongs inside a training pipeline.

- NASA: Transforming Science Prioritization Processes Using Artificial Intelligence: an example of LDA/NLP applied to astrophysics research trends.

- SAO/NASA ADS: astronomy literature discovery.

Publication wording: “I built a reproducible news-text classification and topic-exploration workflow, compared interpretable models against a baseline, and audited leakage and source shortcuts. The project includes held-out evaluation and practical proposals for analyst-assisted decision support.”

Additional corrections to the original interpretation

The original text describes TF-IDF cross-validation, but the active pipeline used LDA topic distributions. It also reports SMOTE improvements even though the displayed code only instantiated SMOTE and never applied it. Word embeddings are claimed in the conclusion but are not implemented. Those claims are removed here.

The original confusion-matrix narrative says 9,980 examples, but its listed counts sum to 8,937 (4,189 + 3,987 + 458 + 303). Counts and interpretation must come from the same execution. This rebuilt notebook renders the actual matrix and derives the headline result from the current metrics.

ROC-AUC of 0.97 does not mean 97% of positive cases were identified. It measures how well scores rank a randomly selected positive-labelled example above a negative-labelled example. Recall at a specified threshold measures the fraction identified.

News text analytics.From patterns to responsible decisions.

News Text Analytics: From Dataset Patterns to Responsible Decisions

1. Import tools

2. Configure paths and reproducibility

3. Record the environment

4. Load the fake-labelled file

5. Load the true-labelled file

6. Validate the schema

7. Attach labels and combine

8. Check missing values

9. Build a conservative article key

10. Audit contradictory labels

11. Remove contradictions, blank bodies and exact copies

12. Prevent repeated titles or bodies crossing splits

13. Inspect metadata shortcuts

14. Make the independent test split

15. Make the validation split

16. Verify split isolation

17. Plot training label balance

18. Summarise training article length

19. Measure a source marker

20. Define transparent text cleaning

21. Apply fixed cleaning

22. Fit a majority baseline

23. Define the TF-IDF representation

24. Build logistic regression

25. Train logistic regression

26. Build and train Naive Bayes

27. Define evaluation metrics

28. Compare validation results

29. Select the model

30. Refit on all development articles

31. Evaluate the independent test

32. Visualise classification errors

33. Assess probability ranking

34. Inspect probability calibration

35. Estimate conditional uncertainty

36. Train a headline-only stress test

37. Count errors without republishing articles

38. Explain logistic regression word associations

Explore topics separately

39. Select a reproducible topic sample

40. Create word counts for LDA

41. Fit a small topic model

42. Inspect topic words

43. Plot average topic weights

44. Generate an evidence-based portfolio conclusion

What this shows a recruiter or client

Business applications — proposals, not measured deployments

Astronomy — a transfer of method

Government — human-reviewed information triage

Limitations and next experiments

Sources and further reading

Additional corrections to the original interpretation