ELI5: How do bots like /user/autotldr read articles and identify which segments are the important ones?

The bot has an FAQ page.

https://np.reddit.com/r/autotldr/comments/31b9fm/faq_autotldr_bot/

SMMRY (pronounced SUMMARY) was created to summarize articles and text.

SMMRY's mission is to provide an efficient manner of understanding text, which is done primarily by reducing the text to only the most important sentences. SMMRY accomplishes its mission by:

• Ranking sentences by importance using the core algorithm. • Reorganizing the summary to focus on a topic; by selection of a keyword. • Removing transition phrases. • Removing unnecessary clauses. • Removing excessive examples.

The core algorithm summarizes in 7 simple steps:

1) Associate words with their grammatical counterparts. (e.g. "city" and "cities") 2) Calculate the occurrence of each word in the text. 3) Assign each word with points depending on their popularity. 4) Detect which periods represent the end of a sentence. (e.g "Mr." does not). 5) Split up the text into individual sentences. 6) Rank sentences by the sum of their words' points. 7) Return X of the most highly ranked sentences in chronological order.

/r/explainlikeimfive Thread