Test Strategy : Automated testing

[planetf1]

We need to consider how we test beehive

• locally
• automatically as part of PR/CI process

• What framework/approach do we want to take with fairly standalone python code? Pytest? Coordinated by poetry?
• How much do we focus on isolated testing (stubbing out LLM for example) vs integrating with LLM?
• If we need LLM, what/where/configuration?
• How do we manage PR/CI verification - generally easy with github actions. Could run a small LLM within a github runner. How do bee work? Can we use same backend for llm? Needs secrets
• How do we check results from tests. With regular code this is easy - assertion of output. However with LLMs involved, so we need another LLM to interpret the results?

This was brought up in issue #125

Originally posted by @planetf1 in #125 (comment)

[psschwei]

For me, the simple first step would be to be make sure the weather example still works (i.e. run that locally after you make your code changes).

As far as testing framework, I think we should sync with @vabarbosa to use the same thing that the bee-py framework uses.