data processing:

* add VIX to some lists
* change create-diff-series to treat all symbols as a single file before scaling
* save list files, not individual series
* scalers are associated with the list

docs/fundementals-to-features.md

@@ -0,0 +1,34 @@
+From the quarterly financial data provided by `yfinance` (i.e., `quarterly_balance_sheet`, `quarterly_cash_flow`, `quarterly_earnings`, `quarterly_financials`, and `quarterly_income_stmt`), you can engineer several key financial ratios and metrics that are valuable for predicting stock price movements. Here’s a breakdown of potential features you can derive from each of these datasets:
+
+### 1. **Quarterly Balance Sheet**
+- **Debt-to-Equity Ratio (D/E)**: Total Liabilities / Shareholder's Equity. This ratio provides insight into the level of a company's debt relative to its equity, indicating financial leverage and risk.
+- **Current Ratio**: Current Assets / Current Liabilities. A measure of liquidity, showing how well a company can meet short-term obligations.
+- **Quick Ratio (Acid Test)**: (Current Assets - Inventories) / Current Liabilities. It's a more stringent test of liquidity compared to the current ratio.
+- **Book Value Per Share**: Total Equity / Number of Outstanding Shares. Indicates the equity value per share and is used to compare the book value with the market value of the shares.
+
+### 2. **Quarterly Cash Flow**
+- **Operating Cash Flow Margin**: Operating Cash Flow / Total Revenue. This ratio measures how much cash a company generates from its operational activities relative to its revenue.
+- **Free Cash Flow (FCF)**: Operating Cash Flow - Capital Expenditures. Free cash flow is an indicator of a company's ability to generate additional revenues.
+- **Cash Flow Coverage Ratios**: Various ratios that use cash flow figures to assess how well a company can meet its financial obligations, such as debt payments.
+
+### 3. **Quarterly Earnings**
+- **Earnings Per Share (EPS) Growth**: Year-over-Year growth in EPS. This is a direct measure of a company's profitability and growth over time.
+- **P/E Ratio**: Market Price Per Share / Earnings Per Share. While market price is not a direct output of `quarterly_earnings`, when combined with current stock prices, this ratio is essential for valuation.
+
+### 4. **Quarterly Financials**
+- **Return on Assets (ROA)**: Net Income / Total Assets. This indicates how efficiently a company uses its assets to generate earnings.
+- **Return on Equity (ROE)**: Net Income / Shareholder's Equity. This measures the profitability of equity investments, indicating how effectively a company uses equity financing.
+- **Gross Profit Margin**: Gross Profit / Total Revenue. Shows the percentage of revenue that exceeds the cost of goods sold.
+
+### 5. **Quarterly Income Statement**
+- **Operating Margin**: Operating Income / Total Revenue. It helps to understand how much of revenue is remaining after subtracting the operating expenses.
+- **Net Profit Margin**: Net Income / Total Revenue. This ratio shows how much of each dollar earned by the company is translated into profits.
+- **Year-over-Year Revenue Growth**: Comparison of revenue from the same quarter in previous years to measure business growth.
+
+### Additional Considerations
+When creating features for a machine learning model:
+- **Normalization**: Consider normalizing or standardizing your financial ratios to avoid scale issues, especially when different features range over different scales.
+- **Lagged Variables**: Use lagged versions of these ratios to capture the financial state in previous quarters, helping to observe trends and cyclicality without introducing look-ahead bias.
+- **Differential Features**: Create features that capture changes from one quarter to the next to identify growth trends or reversals in financial health.
+
+Engineering these features properly allows your machine learning model to capture a holistic view of the company’s financial health and trends, providing strong predictors for stock price movements.