pingleware-tfidf

Node package for TFIDF which stands for Transform Frequency Inverse Document Frequency which given a textual prose will return the three topic sentences

npm test

will produce the results of

[
  'There was nobody there, but he heard the voice again',
  'He thought he heard a voice but at this moment all he could hear was the wind rustling the leaves of the trees all around him',
  'The choice was red, green, or blue'
]

then using ChatGPT to create a new story from these topic sentences,

create a new story from the following three topic sentences 'There was nobody there, but he heard the voice again',
  'He thought he heard a voice but at this moment all he could hear was the wind rustling the leaves of the trees all around him',
  'The choice was red, green, or blue'

will result in,

Title: The Whispering Woods

In the heart of the dense forest, where sunlight struggled to penetrate the thick canopy of ancient trees, a lone figure stood, surrounded by the haunting silence of the wilderness. His name was Ethan, a young explorer with a passion for uncovering the secrets of nature's most enigmatic realms.

As Ethan trekked deeper into the woods, an eerie sensation crept over him. He thought he heard a voice, faint and distant, echoing through the rustling leaves. Pausing in his tracks, he strained his ears, but all he could discern was the gentle whisper of the wind among the branches.

Undeterred, Ethan pressed on, his curiosity driving him forward. There was nobody there, yet he heard the voice again, clearer this time, as if beckoning him deeper into the heart of the forest.

Guided by the mysterious voice, Ethan stumbled upon a clearing, bathed in an otherworldly glow. At its center stood three pedestals, each adorned with a single object—a shimmering gem of vibrant hue.

Approaching cautiously, Ethan realized the significance of the choice before him. The voice urged him to decide: red, green, or blue. Each gem seemed to pulsate with its own unique energy, offering a path into the unknown.

With a mixture of trepidation and excitement, Ethan reached out, his hand hovering over the glowing gems. A decision weighed heavily upon him, for he knew that his choice would shape the course of his journey.

After a moment's hesitation, Ethan's fingers closed around the cool surface of the green gem. As he held it aloft, a surge of power coursed through him, filling him with newfound resolve.

With the echo of the voice fading into the depths of the forest, Ethan embarked on a quest unlike any other, guided by the whispers of the woods and the boundless mysteries that awaited him.

Installation

npm i @pingleware/tf-idf

Usage

"use strict"

const TFIDF = require("@pingleware/tf-idf");

//
// Generated by https://randomwordgenerator.com/paragraph.php
//
const prose = `The irony of the situation hadn't escaped her. She had taken years to sculpt the perfect persona with the perfect look that she shared on Instagram. She knew her hundreds of thousands of followers envied that life she showed and stayed engaged with her because they wanted that life too. The truth was that she wanted the perfect life she portrayed more than any of her fans. The fact was that despite all the perfection she shared on social media, her life was actually more of a mess than most.
Was it a whisper or was it the wind? He wasn't quite sure. He thought he heard a voice but at this moment all he could hear was the wind rustling the leaves of the trees all around him. He stopped and listened more intently to see if he could hear the voice again. Nothing but the wind rustling the leaves could be heard. He was about to continue his walk when he felt a hand on his shoulder, and he quickly turned to see who it was. There was nobody there, but he heard the voice again.
The choice was red, green, or blue. It didn't seem like an important choice when he was making it, but it was a choice nonetheless. Had he known the consequences at that time, he would likely have considered the choice a bit longer. In the end, he didn't and ended up choosing blue.`;

const results = TFIDF(prose);

console.log(results);

will result in

[
  'There was nobody there, but he heard the voice again',
  'He thought he heard a voice but at this moment all he could hear was the wind rustling the leaves of the trees all around him',
  'The choice was red, green, or blue'
]

Resources

• https://en.wikipedia.org/wiki/Tf%E2%80%93idf
• https://monkeylearn.com/blog/what-is-tf-idf/
• https://www.capitalone.com/tech/machine-learning/understanding-tf-idf/
• https://www.geeksforgeeks.org/understanding-tf-idf-term-frequency-inverse-document-frequency/
• https://www.learndatasci.com/glossary/tf-idf-term-frequency-inverse-document-frequency/
• https://www.onely.com/blog/what-is-tf-idf/

Release Schedule

| Date | Version | Description | | :--------: | :-----: | --------------- | | 2024-02-23 | 1.0.0 | Initial release | | | | |

End-of-Life

When a piece of software is useful, there should never be an EOL doctrine. The intention for this application is to achieve immortality ;). At some point of time in the future, this project may appear to be dead and abandon. The opposite will be true! When this project reaches that stage, this project has matured to a level where maintenance is minimal (mostly updating to latest version of Node).

Patrick O. Ingle
February 23, 2024