Ghostbuster.app is a sophisticated AI-generated text detection tool designed to identify whether a given document or text has been generated by an artificial intelligence language model. Here is a detailed overview of what the product does and its key features:
Purpose
Ghostbuster.app is created to address the increasing issue of AI-generated text, particularly in contexts such as academic assignments, news articles, and other written content. It helps users determine if a piece of text has been produced by a large language model like ChatGPT or Claude, rather than a human.Key Features
Model-Agnostic Detection
Ghostbuster.app does not require knowledge of the specific AI model used to generate the text. It operates by passing the document through a series of weaker language models, including unigram, trigram, and non-instruction-tuned GPT-3 models. This approach makes it effective for detecting text generated by unknown or black-box models.Multi-Stage Training Process
The tool uses a three-stage training process:- Computing Probabilities: It converts each document into a series of vectors by computing the probability of generating each word under various weaker language models.
- Selecting Features: A structured search procedure is used to select relevant features from these probabilities.
- Classifier Training: The selected features are then used to train a classifier to identify AI-generated content.
High Accuracy
Ghostbuster.app has demonstrated high accuracy in detecting AI-generated text, achieving an F1 score of 99.0 across tested domains. It outperforms other detection models like GPTZero and DetectGPT, especially in out-of-domain scenarios.Robustness to Edits and Variants
The tool is robust against various edits and prompt variants. It maintains high performance even when the text is lightly edited, such as swapping sentences or paragraphs, although repeated paraphrasing or numerous word-level changes can slightly decrease its accuracy.Training Data
Ghostbuster.app is trained primarily on news articles, student essays, and creative writing in American and British English. However, this means it may not perform as well on texts generated in other styles, languages, or by non-native English speakers.Functionality
- Text Analysis: Users can input a document or text, and the tool will analyze it to determine the likelihood that it was generated by an AI.
- Feature Selection: The tool selects relevant features from the computed probabilities to train an effective classifier.
- Classifier Output: The classifier provides a prediction on whether the text is AI-generated or not.
Limitations and Recommendations
- Text Length: Ghostbuster.app is less reliable for texts shorter than 100 words or between 100-250 words.
- Human Verification: It is recommended to use the tool in conjunction with human verification, especially in critical applications such as academic assessments, to avoid incorrect predictions.