Skip to content

Part 1 (Initial Approach)

Initial Approach

So, to start with - I first wanted to see what would happen if you prompted an LLM to write a story without too many modifications. I opened up text-generation-webui and asked a 70b model to write a story set in the Genshin Impact universe, and this didn't really work well.

What worked

  1. The LLM knew most of the important info about the characters.
  2. The grammatical structure was correct.

That's about all that went well...

What didn't work

Oh so many things.

  1. Length - was way too short.
  2. Writing Style - total rubbish. The LLM would write like a history textbook rather than a story.
  3. Pacing - sometimes it spent hundreds of words describing the color of the sky, other times it skipped over entire battle scenes with just a few words.
  4. Word Choice - So many token phrases 'the tension was palpable' or 'as days turned to weeks'. I never want to read those ever again.
  5. Plot - it was all over the place, plot holes and inconsistencies made it really difficult to read.

Probably more that I'm forgetting.

Take away from this approach

LLMs it turns out, struggle at writing directly like this - it seems they need more guidance than just asking it to write a story.

Now, with that said, it's not like LLMs are totally creatively bankrupt - they do have some good ideas and could probably write some good content, but need more guidance. This gave me an idea - what if I instead had the LLM write an outline, then write each chapter from that outline as a separate prompt. Perhaps that would yield better results.

That leads me on to the next section where I tried implementing that.

Outline-based Approach

I went off and implemented approach - firstly, the LLM would be prompted to write an outline based on the user's provided prompt, something like this:

LLMPrompt = f'''
Please write an outline for a fictional novel based on the following prompt.

<UserPrompt>
{UserPrompt}
</UserPrompt>

Make sure to make your outline markdown formatted and keep a clear structure.
'''

Then, the LLM would generate the outline, this would be extracted and then given to the LLM again, with a prompt asking for the number of chapters in a JSON formatted response, so we can iterate and then generate each chapter. Note that this was done with a fresh context - it does not remember generating the previous response, just the standard AI assistant system prompt.

f'''
Given the provided outline, return a JSON formatted response indicating the number of chapters in the outline.

<Outline>
{Outline}
</Outline>

Format your response as follows:

{"NumChapters": <number of chapters here>}

Make sure to only include JSON in your output - nothing else. It will be read by a computer, and will break if you include anything except JSON.
'''

Finally, with the number of chapters now known, I had a for loop that just asked the LLM to generate each chapter one at a time, keeping the context of previous chapters. This was done with the hope that the LLM would remember what it wrote in the past and keep track of it so the chapters would connect. As you'll see in a minute, this didn't really work.

f'''
Please write chapter {ChapterNum} of {TotalChapters} from the provided outline.

<Outline>
{Outline}
</Outline>

'''

Now, this was much better than the first approach, and lead to some interesting results. These were readable - not good, but also not total garbage. I still cringed when reading them though, so it definitely had to get better.

What worked

  1. Length - the length was now 20k+ words, much more in the target range for what I was going for. Ideally we'd get 100k, but 20k is passable for a starting point. My goal was at least 10k.
  2. Plot - there seemed to be some sort of coherent plot now, not total rubbish that was inconsistent constantly. It wasn't really good but it was better than before (sorta).
  3. Characters - the LLM retained it's ability to understand the characters it was writing about.
  4. Grammar - the grammatical structure was still correct.

What didn't work

  1. Writing Style - total rubbish. The LLM would write like a history textbook rather than a story.
  2. Pacing - sometimes it spent hundreds of words describing the color of the sky, other times it skipped over entire battle scenes with just a few words.
  3. Word Choice - So many token phrases 'the tension was palpable' or 'as days turned to weeks'. I never want to read those ever again.
  4. Chapter Consistency - this is a new problem, before it was only writing basically one chapter, but now since we are doing multiple generations, it tended to forget what happened in previous chapters and write totally different and disjointed things
  5. Sanity - sometimes, the LLM went insane and just started writing garbage (infinitely repeating words)

Conclusion

So, with the second approach, I saw better results, but it seemed that the LLM would go insane while writing longer novels about half of the time. I eventually had to play with the models to see if some worked better - which leads me onto my next discovery.

Check out the next page for that!