# Text Insight

> See what is inside any text, in the browser: named entities in context, an entity co-occurrence
> graph with centrality, word and n-gram frequencies, a word cloud, RAKE and TF-IDF keywords,
> sentence-by-sentence sentiment and readability scores, free and with nothing uploaded. An optional paid
> Relations view extracts who-did-what triples with quoted, verified evidence and draws them on the graph.

URL: https://text-insight.skillsafe.ai/
Price: every view is free with no sign-up, except Relations, which needs a SkillSafe sign-in and uses credits
(the estimate is shown as reserved before the run; you pay what the run uses).

## Input
- Paste text, or open/drop a file: TXT, Markdown, PDF (text layer), DOCX, HTML, RTF, CSV.
- Up to 400,000 characters are analysed; entity recognition covers the first 150,000. The app says so when it clips.
- Three built-in examples (a history of the Panama Canal, fictional product reviews, a fictional business memo) cost nothing.

## Views
- Overview: Flesch Reading Ease, Flesch-Kincaid, Gunning Fog, SMOG, Coleman-Liau, ARI, LIX; word, sentence and
  paragraph counts, unique words, lexical diversity (MTLD), reading and speaking time; words-per-sentence chart.
- Entities: people, places, organisations, dates, money and percentages, merged across spellings ("Roosevelt" joins
  "Theodore Roosevelt"), each with every mention shown in its sentence and the entities it appears with.
- Graph: entities linked when they share a sentence (or paragraph). Filters for type, minimum mentions and minimum
  link strength; optional key-term nodes; force layout with drag, zoom and focus. Analytics: nodes, links, density,
  clusters, average degree, and per-node degree and betweenness centrality.
- Words: frequencies with stopword removal, custom stopwords, minimum length and number handling; two- and
  three-word phrases; word cloud.
- Keywords: RAKE key phrases and TF-IDF distinctive terms (each paragraph is one document).
- Sentiment: VADER-method score per sentence (-1 to +1), the overall tone, the words pulling it up or down, and the
  most positive and negative sentences.
- Text: the full text with entities highlighted by type and optional sentence tinting by tone.
- Relations (paid, sign-in): one part of the text (whole paragraphs, at most 12,000 characters; longer texts are
  split into parts and each part is run separately) plus the entity list found in it goes to an AI model
  (gpt-terra). The page shows exactly what is sent and the reserved cost first. The model returns
  subject-relation-object triples, each with the supporting text quoted. The page then checks every quote word
  for word against the source and every subject and object against the entity list or the text:
  "verified" (quote found, both ends supported and named in the quote), "flagged" (supported, but an end is only
  implied, e.g. a pronoun, or the quote uses an ellipsis) or "dropped" (quote not in the text, or a name found
  nowhere). Kept triples become labelled arrows on the graph. The built-in examples have free saved replies.

## Exports
JSON of the whole analysis; CSV of entities, entity mentions, word frequencies, n-grams, keywords and sentences with
sentiment; graph nodes and edges as JSON (with relation arrows when present); relations as CSV or JSON; graph as PNG or SVG; word cloud as PNG; a Markdown summary report.

## How it works
Everything except Relations runs client-side in the page; Relations sends only the part you run. Entity recognition uses the compromise NLP library and its dates plugin
(MIT). Sentiment uses the VADER lexicon (Hutto and Gilbert, MIT) with negation, intensifier, contrast and emphasis
rules. PDF and DOCX reading uses the same in-browser parsers as the SkillSafe document converter. Results are
automatic: names can be missed or mislabelled, and readability formulas are estimates for English prose.

## API
The Relations run can be called from code with a SkillSafe token: https://text-insight.skillsafe.ai/api.html
(input fields, free estimate, run with an Idempotency-Key, the output contract). The free views have no API.