Skip to content

file_formats.md

Latest commit

 

History

History
136 lines (107 loc) · 10.5 KB

file_formats.md

File metadata and controls

136 lines (107 loc) · 10.5 KB
Raw

File Formats

This document describes the common file formats used by the Novel Writer application for storing parameters, intermediate generation steps, and final outputs. Most files are stored in the user-defined output directory (which defaults to current_work/ relative to the application's execution path).

General File Types

  • .txt files: Primarily used for the parameters.txt file, storing key-value settings. Can also be used for simple plain text if needed elsewhere.
  • .json files: Used for structured data such as character profiles (characters.json) and faction details (factions.json). These files are typically pretty-printed with an indent of 4 for human readability.
  • .md (Markdown) files: The most common format, used for narrative content, outlines, lore, LLM prompts, and final prose. Markdown allows for easy formatting, readability, and can be readily converted to other formats if needed.

Specific File Descriptions

Below are descriptions of key files generated and used by the application.

Configuration and Core Data

  1. parameters.txt

    • Purpose: Stores all user-defined settings from the "Novel Parameters" tab. This file is crucial as it guides all subsequent generation processes.
    • Format: Text file with key-value pairs, one per line (e.g., Novel Title: The Galactic Scourge).
    • Key Fields: Includes Output Directory, Genre, Subgenre, Story Length, Story Structure, Novel Title, Author Name, Theme, Tone, Gender Generation Bias, and any dynamic parameters specific to the selected genre/subgenre.
    • Generated by: parameters.py (Parameters class).
    • Used by: Almost all other modules in the application.

  2. generated_lore.md

    • Purpose: Contains the main narrative lore and background for the story's universe. This is typically generated by an LLM based on initial parameters and summaries of characters/factions.
    • Format: Free-form Markdown text. Includes details about history, culture, technology, significant events, key locations, etc.
    • Generated by: lore.py (Lore class, generate_lore() method).
    • Used by: story_structure.py, scene_plan.py, and chapter_writing.py as primary contextual information.

  3. characters.json

    • Purpose: Stores detailed profiles for all generated characters.
    • Format: JSON. A root object usually with a "characters" key, which holds a list of character objects (dictionaries).
    • Character Object Keys: name, role (e.g., "protagonist"), gender, age, title, occupation, faction, faction_role, homeworld, home_system, appearance_summary, personality_summary, goals (list), motivations (list), flaws (list), strengths (list), family (a sub-dictionary with parents, siblings, spouse, children), backstory_summary, arc.
    • Generated by: Initial data from Generators/CharacterGenerator.py; potentially enriched by lore.py.
    • Used by: lore.py, story_structure.py, scene_plan.py, chapter_writing.py.

  4. factions.json

    • Purpose: Stores detailed profiles for generated factions.
    • Format: JSON. A list of faction objects (dictionaries).
    • Faction Object Keys: faction_name, faction_profile, primary_traits (list), government_type, economic_model, military_strength_description. The file will also contain genre-specific location data. For example, Sci-Fi might include: systems (list of system objects, each with name, star_type, habitable_planets).
    • Location Object Keys (example for Sci-Fi Planets within habitable_planets): name, population, climate, resources, strategic_value, is_capital. Other genres will have different keys for their specific location types (e.g., regions and cities for Fantasy, territories for Western).
    • Generated by: The relevant genre handler in Generators/GenreHandlers/ (e.g., scifi_handler.py, fantasy_handler.py).
    • Used by: lore.py, story_structure.py, chapter_writing.py.

Character and Story Arc Development Files

  1. background_[role]_[name].md files (e.g., background_protagonist_jax_ryker.md)

    • Purpose: Stores the detailed, LLM-generated backstory for a main character (protagonist, deuteragonist, antagonist).
    • Format: Free-form Markdown text.
    • Generated by: lore.py (main_character_enhancement() method).
    • Used by: story_structure.py (generate_arcs() method).

  2. character_arcs.md

    • Purpose: Contains LLM-generated narrative arcs outlining the development and progression for the main characters.
    • Format: Markdown text.
    • Generated by: story_structure.py (generate_arcs() method).
    • Used by: story_structure.py (generate_faction_arcs() method as context).

  3. faction_arcs.md

    • Purpose: Contains LLM-generated story arcs for major factions, aligned with the selected story structure.
    • Format: Markdown text.
    • Generated by: story_structure.py (generate_faction_arcs() method, first LLM call).
    • Used by: story_structure.py (generate_faction_arcs() for reconciliation step).

  4. reconciled_arcs.md

    • Purpose: Presents a unified story arc that integrates both character and faction arcs, following the selected story structure.
    • Format: Markdown text.
    • Generated by: story_structure.py (generate_faction_arcs() method, second LLM call).
    • Used by: story_structure.py (add_locations_to_arcs() method).

  5. reconciled_locations_arcs.md

    • Purpose: This is the reconciled story arc enhanced with specific locations (e.g., planets for Sci-Fi, cities for Fantasy). It serves as the primary high-level plot document before breaking into detailed acts/sections or chapters.
    • Format: Markdown text.
    • Generated by: story_structure.py (add_locations_to_arcs() method).
    • Used by: story_structure.py (improve_structure() or _outline_short_story_plot()).

Detailed Plot and Outline Files

  1. [structure]_[section].md files (e.g., 6-act_structure_beginning.md, novel_epic_hero_journey_the_ordinary_world.md)

    • Purpose: For long-form works (Novella, Novel, Epic), each file contains a detailed plot breakdown for a specific act or major section of the story, based on the selected structure.
    • Format: Markdown text.
    • Generated by: story_structure.py (improve_structure() method).
    • Used by: scene_plan.py (generate_chapter_outline() method).

  2. plot_short_story_[structure].md files (e.g., plot_short_story_freytags_pyramid.md)

    • Purpose: For short stories, this file contains the detailed plot outline for the entire story, following the selected structure.
    • Format: Markdown text.
    • Generated by: story_structure.py (_outline_short_story_plot() method).
    • Used by: scene_plan.py (_plan_short_story_scenes() method).

  3. chapter_outlines_[structure]_[section].md files (e.g., chapter_outlines_6-act_structure_rising_action.md)

    • Purpose: For long-form works, these provide chapter-by-chapter outlines for a specific act/section. They suggest scenes, characters, and locations for each chapter within that section.
    • Format: Markdown text, typically with headings for each chapter.
    • Generated by: scene_plan.py (generate_chapter_outline() method).
    • Used by: scene_plan.py (_plan_long_form_scenes() method).

Scene Plan Files

  1. detailed_scene_plans/scenes_[structure]_[section]_ch[GlobalNum].md files (e.g., detailed_scene_plans/scenes_6-act_structure_beginning_ch1.md)

    • Purpose: For long-form works, each file contains the detailed scene-by-scene plan for one single chapter.
    • Format: Markdown text. Scenes are usually denoted by ## Scene X: Title or similar headings, followed by descriptions (setting, characters, actions, dialogue notes).
    • Location: Stored in the detailed_scene_plans/ subdirectory within the main output directory.
    • Generated by: scene_plan.py (_plan_long_form_scenes() method).
    • Used by: chapter_writing.py (write_chapter() method) as direct input for prose generation.

  2. scenes_short_story_[structure].md files (e.g., scenes_short_story_3-act_structure.md)

    • Purpose: For short stories, this file contains the detailed scene-by-scene plan for the entire story.
    • Format: Markdown text, with scene headings (e.g., ## Scene X: ... or ### Scene X - ...) and descriptions.
    • Generated by: scene_plan.py (_plan_short_story_scenes() method).
    • Used by: chapter_writing.py (_write_short_story_prose() method).

Final Prose Output Files

  1. chapters/chapter_[num].md files (e.g., chapters/chapter_1.md)

    • Purpose: For long-form works, these files contain the final LLM-generated narrative prose for each chapter.
    • Format: Markdown text.
    • Location: Stored in the chapters/ subdirectory within the main output directory.
    • Generated by: chapter_writing.py (write_chapter() method).

  2. prose_short_story_[title].md files (e.g., prose_short_story_the_last_signal.md)

    • Purpose: For short stories, this file contains the final LLM-generated narrative prose for the entire story.
    • Format: Markdown text.
    • Generated by: chapter_writing.py (_write_short_story_prose() method).

Logging and Prompts

  1. prompts/ subdirectory

    • Purpose: This directory contains numerous Markdown (.md) files that are logs of the exact prompts sent to the LLM for various generation steps (e.g., lore generation, arc creation, scene planning, prose writing).
    • Format: Markdown text, reflecting the prompt content.
    • Generated by: Various methods across different modules, typically using helper_fns.save_prompt_to_file().
    • Use: Extremely useful for debugging, understanding LLM behavior, and refining prompt engineering.

  2. app.log

    • Purpose: The main application log file, recording events, warnings, and errors during execution.
    • Format: Plain text, with each line typically containing a timestamp, log level, module name, and the log message.
    • Generated by: The logging setup in logger_config.py and used via app.logger throughout the application.