Книга: Algorithmic Short Selling With Python
Назад: Chapter 9: Asset Allocation
Дальше: Chapter 11: Unlock Access to the Code Bundle and the PDF Version

10

The Trading Journal

"We do not trade the markets. We trade our beliefs about the markets."

– Van Tharp

This book has spent nine chapters on the mechanics of algorithmic short selling: regime detection, position sizing, universe refinement, the full toolbox of long/short strategies. All of that is the hardware. This chapter is about the software, and specifically about the programmer running it.

Debugging code is, by now, a largely solved problem. Paste the error into an artificial intelligence (AI) and the fix comes back in seconds. In fact, the code for this chapter was written by AI from an existing highly customized Excel file. So, if debugging the program has never been easier, how do we debug the programmer?

Our beliefs, biases, and emotional patterns do not appear in stack traces. They reveal themselves slowly, over hundreds of trading days, in the gap between what the system said and what we actually did. The trading journal is the instrument that makes that gap visible, quantifiable, and correctable.

This chapter builds a complete, practical trading journal on top of Airtable, combining two systems that most traders treat separately. Part A automates the analytics side: every evening, it reconciles the order file against the broker execution report, classifies every discrepancy, and writes the full audit trail to Airtable. Part B automates the psychology side: a two-minute morning form and a five-minute evening form feed a gamified scoring system that tracks the mental performance over time. An AI layer sits on top of both, reading the journal entries as a trading psychologist would.

In this chapter, we will cover the following topics:

Technical requirements

The code in this chapter requires Python 3.9 or later with the requests and pandas packages installed. All Airtable communication uses the REST API directly via requests, so no Airtable SDK is needed. Credentials are stored in a .env file at the project root containing three keys:

The notebook file is trading‑journal‑v5.ipynb in the companion repository.

The trader's mental software

"Until you make the unconscious conscious, it will direct your life and you will call it fate."

– attributed to Carl Jung

Building a profitable algorithmic edge requires two things to align: a strategy with a positive expected value, and a trader psychologically capable of executing it without interference. Most books address the first condition in detail and treat the second as an afterthought. In practice the second condition fails far more often, and the consequences are just as severe.

The core problem is that our subconscious beliefs run the show. They operate below the level of conscious reasoning, shaping our perception of risk, our tolerance for uncertainty, and our willingness to act or refrain from acting. No amount of system-building resolves this if the beliefs themselves are misaligned with the strategy. Below are common beliefs market participants have encountered. This list is not exhaustive.

The six beliefs that cost traders money

The need to be right is the most destructive belief for a systematic trader. Arbitrageurs are paid to be right about relative value; trend followers are paid precisely for the discomfort of being wrong most of the time. A trend-following trader who collapses taking a loss (process) with being wrong (identity) will cut winners too early and hold losers too long, inverting the strategy's payoff profile entirely.

The need for action produces low-quality, impulsive positions. Markets spend long periods doing nothing useful for a systematic short seller. It is not because we show up at our desk that we should be trading. The urge to trade, to feel productive, to justify the screen time, generates mirages. It takes wisdom to recognize that not trading is an important trading decision.

The need for validation attaches traders to narratives. A stock is obviously overvalued; the thesis is compelling; the research is solid. When the price rises anyway, the trader adds rather than cuts, because the story feels more real than the mark-to-market loss. Story, and more importantly the need to be right, supersede the imperative to make money.

Loss aversion, formalized by Kahneman and Tversky, drives the disposition effect: selling winners too early and holding losers too long. For short sellers, this asymmetry is especially dangerous. Coupled with a scarcity mindset, loss aversion prolongs the agony of market participants during short squeezes.

The need for certainty shows up as perfectionism: waiting for the perfect setup to enter or close a position. Markets are probabilistic. Profitable trading requires making decisions under uncertainty and living with the discomfort. Every box ticked has a price tag attached to it. So, if we wait until all the boxes are ticked, we may show up only after shorts become crowded. Certainty is an expensive luxury on the markets.

The locus of control, finally, determines how traders respond to losses. An external locus of control produces blame: "they", the analysts, the price action etc. An internal locus of control takes responsibility for everything: how can I improve tomorrow? Over a large sample, the difference in outcomes between these two dispositions is hardly subtle. Successful traders focus on what they can control. They cannot manifest a trending market, but they can certainly manage their exposures as we saw in Chapter 8. As long as we shift responsibility toward external circumstances, we put on the victim role and delay our learning process. Victims do not build empires. The journal is the tool that reinforces the internal locus, one entry at a time. It is designed to keep us focused on what is within our control and therefore perfectible.

The journal as a debugging tool

Trading is a business. Successful businesses keep meticulous ledgers. Without precise records, decisions are made on feelings rather than data. The trading journal is that ledger. It records what was supposed to happen, what happened, and the emotions about the gap between the two.

For short sellers, the case is even stronger. The short side is less forgiving. The market drifts upward over time. Continuous improvement through Kaizen is not a self-help abstraction; it is a survival mechanism. The compound effect of small daily improvements is a path to trading mastery.

A journal keeps market participants accountable. Market participants often underestimate the drift from the lack of adherence to their algorithms. A well-maintained journal will keep score of stops overridden, position sizes renegotiated, and impulse trades.

Our subconscious beliefs bypass conscious decision making. Self-deception is a built-in feature that covers its own tracks. Recurring thought patterns, emotional triggers, and cognitive biases only become visible across dozens of entries. Once we become aware of our biases, we cannot unsee them. We have all dismissed that little voice inside our head that whispers "I am not good enough" or "I don't belong on the markets" after a bad trade. Repeat that entry 20 times, and we cannot ignore those symptoms of toxic shame anymore.

If we do not analyze and reflect on our trades, we are bound to repeat the same mistakes. The journal converts anecdote into pattern, and pattern into correctable behavior.

Two types, one system

There are two kinds of trading journal, and a complete system needs both.

The analytics journal is the trade log. It records the objective facts of every trade: ticker, direction, entry and exit prices, quantity, commission, gross and net profit and loss. It also records the trades that did not happen: a) the signals generated by the system that we chose not to act on (missed trades), and b) the fills that arrived without a prior signal (overrides). These three categories contain the most useful information in the entire dataset.

The psychology journal records the subjective facts of every trading day: our mindset before the open, our confidence level, whether we completed the pre-market checklist, the directional forecast and the reasoning behind it, our reflection in the evening, what went well, and how we intend to improve tomorrow. Three gratitude entries prime the brain for a positive outlook. A single Kaizen thought converts to an actionable improvement. Here is an important nuance. We naturally tend to conflate making money with being right. Conversely, we tend to assign making losses with being wrong. Trading right is doing the right thing even if it means losing money. One powerful way to reframe losses is to remind ourselves that "sometimes we win, sometimes we learn, and today I am grateful for the lesson." The design of this system prioritizes reducing friction to the minimum viable dose. The analytics journal is fully automated: the order file is generated from strategies, brokers send the execution reports, and the notebook does the matching, classification, and Airtable updates. The psychology journal runs on your phone via two Airtable form links, bookmarked for before the open and after the close.

This automated journal has a powerful added feature few traders ever bother to track. It takes its roots in World War II.

The Abraham Wald principle

During the Second World War, Allied forces studied bullet-hole distributions on aircraft returning from bombing missions. They wanted to reinforce airplanes but they had to make choices because of the added weight. Damage was concentrated on the fuselage and wings. The intuitive conclusion was to reinforce the areas with the highest density of bullet impacts.

Figure 10.1: Survivorship bias plane (source: Wikipedia Survivorship Bias)

Figure 10.1: Survivorship bias plane (source: Wikipedia Survivorship Bias)

Abraham Wald, a statistician working with the Statistical Research Group, saw the data sampling error. The aircraft being examined were the survivors. The holes they carried were evidence of damage that was survivable. The areas with no holes, particularly the engines, and cockpit, were not undamaged. They were the locations where damage caused the plane not to return at all. To protect future aircraft, the analysis needed to focus on the planes that never came back.

Missed trades are the planes that never came back. Every time the system generated a signal that was ignored, because the stock felt too risky, the position size felt too large, or the market felt wrong, that decision is invisible in the standard trade log. The log only shows executed trades. It is the surviving fleet, and it tells only half the story.

When missed and executed trades are logged alongside and their distribution studied by motive, patterns emerge. Perhaps we skip a few short entries after a string of losses, because our confidence has been shaken. Perhaps we resize positions after a winning or losing streak. Perhaps, we override a stop loss here or there. Like the holes in the fuselage, the blanks in our strategies tell us a lot about who we are.

Making journaling stick

"The chains of habit are too weak to be felt until they are too strong to be broken."

– Warren Buffett

Journaling initiatives fail for three reasons: friction, delayed gratification, and an excessive focus on failure. This system addresses all three.

Friction is eliminated wherever possible. The analytics journal is automated: no manual data entry, no copying prices from broker screens. The psychology journal uses phone-optimized Airtable forms with mostly dropdown and numeric fields. Pre-populated defaults (session = "pre" on the morning form, session = "post" on the evening form) reduce the number of decisions required to zero.

Reward is built in through gamification. Each day's psychology journal is scored out of 120 points based on completeness. Consecutive completion days build a streak, which generates a multiplier compounding at 1 + streak/20: by day 20 the multiplier is 2.0x; by day 40 it is 3.0x. The gamified score is computed automatically each time Part B runs and patched back to Airtable, so the dashboard reflects current standing without any manual calculation. Now, here is a little tip. Decide on a reward once you reach 1,000, 2,000, 3,000 points. That is the textbook definition of a Specific, Measurable, Achievable, Relevant, Time-Bound (SMART) goal. In a very successful hedge fund I worked at, the principal would buy a bottle of wine that was a multiple of the performance every Friday.

The focus is on progress and positive outcome, not failure. The morning form asks what is supposed to happen today and why. The evening form asks what went well and how to improve tomorrow. The gratitude entries prime the brain to search for positive signals. The AI prompts in the final section of this chapter extend this design by reading the journal as a coaching conversation, not an audit.

Setting up Airtable

Airtable was chosen for three reasons: a) it exposes a clean REST API that Python can call without a dedicated SDK, b) it renders as a familiar spreadsheet for traders who live in Excel, and c) its form feature converts any table into a mobile-ready data entry screen without a single line of front-end code. The free tier handles the volumes a systematic trader generates without difficulty.

This chapter has two distinct files. One sets up the Airtable, and the other is about the TradeLog journal.

Creating the TradeLog table

Create a new Airtable base and inside it a table named TradeLog (case-sensitive: the notebook reads this from the AIRTABLE_TABLE environment variable). Airtable Omni AI will graciously offer to create a TradeLog for you. This will interfere with what we are building, so no need for that. Add the following fields:

Field

Type

Purpose

Ticker

Single line text

Primary field. Stock symbol.

entry_date

Date

Date the position was opened.

exit_date

Date

Date closed. Empty means open.

Strategy

Single line text

Strategy label. Position key with ticker.

Broker

Single line text

Executing broker.

order_type

Single line text

limit, market, stop, etc.

quantity_order

Number

Intended quantity (signed).

quantity_exec

Number

Executed quantity (signed).

price_order

Number

Limit or target order price.

price_exec

Number

Actual fill price.

price_exit

Number

Exit fill price when closed.

stop_price

Number

Stop loss level.

target_price

Number

Profit target level.

risk_pct

Number

Risk as % of portfolio at entry.

commission

Number

Total commission for this leg.

gross_pnl

Number

Gross profit/loss on the trade.

net_pnl

Number

Net P&L after commission.

trade_result

Single line text

win, loss, or breakeven.

missed_trade

Single line text

missed or override; blank = normal.

record_id

Single line text

Broker execution reference.

Table 10.1: Airtable TradeLog fields

Airtable's Meta API does not support autoNumber as a primary field. Once the table is set up, we recommend you manually add autoNumber as a primary field. Name this column tradeID. Replace the existing ticker column. It will prompt a message and replace it as the primary field.

Next, let's create the psychology journal.

Creating the journal table

Add a second table named journal (lowercase) to the same base. This table stores one row per session per trading day: a pre row from the morning form and a post row from the evening form.

Field

Type

Purpose

date

Date

Primary field. Trading date.

session

Single line

pre or post.

mindset_pre

Number

1 to 5. Pre-market fear/greed scale.

confidence_pre

Number

1 to 5. Pre-market confidence.

checklist_done

Number

1 if checklist is complete, 0 otherwise.

perf_forecast

Number

Predicted session performance (%).

bc_1

Long text

First reason supporting the forecast.

bc_2

Long text

Second reason supporting the forecast.

mindset_post

Number

1 to 5. Post-market fear/greed scale.

confidence_post

Number

1 to 5. Post-market confidence.

perf_actual

Number

Actual session performance (%).

what_went_well_1

Long text

First positive observation.

what_went_well_2

Long text

Second positive observation.

what_went_well_3

Long text

Third positive observation.

tomorrows_kaizen

Long text

One concrete improvement for tomorrow.

notes

Long text

Free-form journal entry.

gratitude_1

Long text

First gratitude entry.

gratitude_2

Long text

Second gratitude entry.

gratitude_3

Long text

Third gratitude entry.

bulls_eye

Number

Computed: sign(forecast x actual).

score

Number

Computed: completeness score 0 to 120.

streak

Number

Computed: consecutive completion days.

multiplier

Number

Computed: 1 + streak/20.

final_score

Number

Computed: score x multiplier.

Table 10.2: Airtable journal fields

Airtable's Meta API does not support autoNumber as a primary field. Once the table is set up, we recommend you manually add autoNumber as a primary field. Name this column entry. Replace the existing ticker column. It will prompt a message and replace it as the primary field.

Next, let's create the forms.

Creating the forms

The journal table requires two form views: morning check-in and evening review. Airtable form views cannot be created programmatically via the Meta API; they must be created once in the Airtable UI:

  1. Open the journal table | click + Add a view | select Form | name it Morning Check-in.
  2. In the form editor, hide all fields and then show only: date, session, mindset_pre, confidence_pre, checklist_done, perf_forecast, bc_1, bc_2.
  3. Click Share form | copy the link | bookmark it on your phone. This is your two-minute morning ritual.
  4. Repeat for a second form named Evening Review, showing: date, session, mindset_post, confidence_post, perf_actual, what_went_well_1, what_went_well_2, what_went_well_3, tomorrows_kaizen, notes, gratitude_1, gratitude_2, gratitude_3.
  5. Click Share form | copy the link. This is your five-minute evening ritual.

Form views survive table resets (record deletions). Only deleting the table itself removes them. Paste your form links somewhere permanent before running the setup notebook for the first time.

Let's go back to the fields and explain some of them:

Next, let's proceed to the evening trade reconciliation. Share each form link and bookmark it on your phone. The forms survive table resets; only deleting the table itself destroys them. Airtable's Meta API does not support creating form views programmatically, so this step must be done manually in the UI once.

Part A: trade reconciliation

Part A runs once each evening after the session closes. It compares the order file against the broker execution report, classifies every outcome, updates the live portfolio CSV, and writes the full audit trail to Airtable. The entire workflow lives in ten notebook cells.

The three possible outcomes of reconciliation are:

Next, let's proceed to writing the .env file and then set up the credentials.

Creating your .env file

This notebook reads credentials from a .env file in the same folder as this notebook. Create it once — you never need to touch it again. Here is a step-by-step process to create and store your credentials in a .env file.

Step 1: Get your Airtable credentials:

Key

Where to find it

AIRTABLE_API_KEY

airtable.com | account icon | Developer Hub | Personal access tokens | Create token (scopes: data.records:read, data.records:write)

AIRTABLE_BASE_ID

Open your base | look at the browser URL: — the segment starting with app

AIRTABLE_TABLE

The exact name of your TradeLog table (default: TradeLog)

Table 10.3: .env file set-up

Step 2: Create the file in Notepad:

  1. Open Notepad (Windows) or TextEdit in plain-text mode (Mac).
  2. Paste the three lines below, replacing the placeholders with your real values:
    1. AIRTABLE_API_KEY=your_personal_access_token_here
    2. AIRTABLE_BASE_ID=appXXXXXXXXXXXXXX
    3. AIRTABLE_TABLE=TradeLog

Step 3: File | Save As:

Add .env to your .gitignore — it contains a secret token and must never be committed.

This one-time set-up is a standard procedure to keep credentials hidden from the public. Next, we will load the credentials saved in this .env file.

Cell 1: setup

We will be communicating with the Airtable API. We therefore need to load the credentials. The notebook begins by loading the three required environment variables from a .env file and verifies all three required keys are present. Using os.environ.setdefault means variables already exported in the shell are never overwritten.

import os, requests, pandas as pd  from pathlib import Path    def load_env(path='.env'):      if not Path(path).exists():          raise FileNotFoundError('No .env file found.')      with Path(path).open() as f:          for line in f:              line = line.strip()              if not line or line.startswith('#') or '=' not in line: continue              k, _, v = line.partition('=')              os.environ.setdefault(k.strip(), v.strip())    load_env()  missing = [k for k in ('AIRTABLE_API_KEY','AIRTABLE_BASE_ID','AIRTABLE_TABLE')             if not os.getenv(k)]  if missing: raise EnvironmentError(f'Missing in .env: {missing}')  print('credentials OK')

Loading credentials from .env into the process environment so all subsequent cells can call the Airtable API. Let's briefly explain what the code does:

  1. Importing standard libraries (os, requests, pandas, pathlib, datetime).
  2. loading_env opens .env, skips comments and blank lines, and sets each KEY=VALUE pair via os.environ.setdefault, existing shell vars are never overwritten.
  3. Calling load_env(), then verifying the three required keys are present; raise an error if any are missing.

Next, we define the Airtable helpers.

Cell 2: Airtable helpers

Three functions handle all Airtable communication; they are:

Both at_append and at_patch accept an optional url parameter that defaults to the TradeLog endpoint. Part B passes AT_J_URL to reuse the same functions for the journal table.

BASE_ID    = os.getenv('AIRTABLE_BASE_ID')  AT_URL     = f"https://api.airtable.com/v0/{BASE_ID}/{os.getenv('AIRTABLE_TABLE')}"  AT_J_URL   = f'https://api.airtable.com/v0/{BASE_ID}/journal'  AT_HEADERS = {'Authorization': f"Bearer {os.getenv('AIRTABLE_API_KEY')}",                'Content-Type': 'application/json'}    PORTFOLIO_COLS = ['ticker','position','cost','stop_price','target_price','strategy','entry_date']    def at_list_open(ticker, strategy):      formula = f"AND({{ticker}}='{ticker}', {{strategy}}='{strategy}', NOT({{exit_date}}))"      r = requests.get(AT_URL, headers=AT_HEADERS,                       params={'filterByFormula': formula,                               'sort[0][field]': 'entry_date',                               'sort[0][direction]': 'asc'})      r.raise_for_status()      return r.json().get('records', [])    def at_append(fields, url=None):      url   = url or AT_URL      clean = {k: v for k, v in fields.items()               if v is not None and not (isinstance(v, float) and pd.isna(v))}      r = requests.post(url, headers=AT_HEADERS, json={'fields': clean})      r.raise_for_status(); return r.json()    def at_patch(record_id, fields, url=None):      url = url or AT_URL      r   = requests.patch(f'{url}/{record_id}', headers=AT_HEADERS, json={'fields': fields})      r.raise_for_status(); return r.json()

Building the Airtable endpoint URLs and defining the three API operations the rest of the notebook relies on. Let's explain the code:

This defines three reusable functions that encapsulate all Airtable HTTP calls. Both at_append and at_patch accept an optional url so Part B can reuse them for the journal table.

Now, we load the csv files.

Cell 3: load_csv

This function returns an empty DataFrame when the file does not yet exist, so the notebook runs cleanly on day one before any CSV has been written.

def load_csv(path):      p = Path(path)      return pd.read_csv(p) if p.exists() else pd.DataFrame()

This function provides a safe csv loader that does not crash if no file is found.

Let's go through the simulated sample data.

Cell 4: sample data

We have created various simulations to cover frequently used cases. The notebook covers six scenarios using fictitious tickers and brokers (IronBridge Capital and NexusTrade Securities). In production, replace the sample DataFrames with load_csv calls pointing at the broker export files.

The table below covers six reconciliation scenarios: add, close in full, partial exit, reversal, missed, and override.

Ticker

Case

Description

KROX

Build

Pyramid short: adding a fourth tranche to an existing three-lot position.

VEGA

Full close

Short entered three weeks ago, exited in full today with a profit.

PULS

Partial exit

Two-lot pyramid short: FIFO closes only the oldest lot.

FLUX

Reversal

Short -150 shares; buy +300 closes the short and opens a new long.

NOVA

Missed

Order placed, never filled. The Abraham Wald signal.

ORION

Override

Fill with no prior order: reactive, unplanned trade.

Table 10.4: Six reconciliation scenarios

Now, we need to seed the portfolio in the Airtable TradeLog.

Cell 5: _seed_portfolio

When the TradeLog is empty but the portfolio is not, _seed_portfolio writes every existing position as an opening entry in Airtable. It checks for existing open lots first, so it is safe to call multiple times. Positions built across multiple lots must be passed as separate portfolio rows, each with its own entry_date, so that the FIFO handler can consume them oldest-first.

def _seed_portfolio(portfolio):      for _, row in portfolio.iterrows():          ticker, strategy = row['ticker'], row['strategy']          if at_list_open(ticker, strategy):              print(f'  SKIP   {ticker}/{strategy}  already seeded')              continue          at_append({              'entry_date': row['entry_date'], 'ticker': ticker,              'strategy':   strategy,              'quantity_exec': int(row['position']),              'price_exec':   round(float(row['cost']), 4),              'stop_price':   row.get('stop_price'),              'target_price': row.get('target_price'),          })          print(f'  SEEDED {ticker}/{strategy}  pos={int(row["position"]):+d} @ {float(row["cost"]):.4f}')    # Run once to initialise, then comment out  _seed_portfolio(sample_portfolio)

Populating an empty TradeLog with pre-existing positions.

Every portfolio row exists as an open lot in Airtable. We only need to run this cell once after initial setup. It is safe to call again as no duplicates are created.

Next, we classify trades into three categories.

Cell 6: _split_trades

A single pd.merge with how='outer' and indicator=True replaces three separate loops. The merge key is (date, ticker, broker). The _merge column labels each row as both, left_only (missed), or right_only (override).

def _split_trades(orders, executions):      merged = pd.merge(          orders, executions,          on=['date','ticker','broker'],          how='outer', suffixes=('_ord','_exec'), indicator=True      )      matched  = merged[merged['_merge'] == 'both'].copy()      missed   = merged[merged['_merge'] == 'left_only'].copy()      override = merged[merged['_merge'] == 'right_only'].copy()      return (matched.drop(columns='_merge'),              missed.drop(columns='_merge'),              override.drop(columns='_merge'))

Classifying every order and execution into one of three categories using a single merge.

This generates three clean dfs: matched, missed, override.

Next, we focus on writing trades missed and override.

Cell 7: _write_missed and _write_override

Neither function modifies the portfolio CSV. Both write a pure audit record to the TradeLog, tagged with the missed_trade field so they can be filtered and studied separately.

def _write_missed(missed):      for _, row in missed.iterrows():          at_append({'entry_date': row['date'], 'ticker': row['ticker'],                     'strategy': row.get('strategy'), 'broker': row['broker'],                     'quantity_order': int(row['quantity']),                     'price_order': row.get('price'), 'missed_trade': 'missed'})          print(f"  MISSED   {row['ticker']}  qty={int(row['quantity']):+d}")    def _write_override(override):      for _, row in override.iterrows():          at_append({'entry_date': row['date'], 'ticker': row['ticker'],                     'broker': row['broker'],                     'quantity_exec': int(row['quantity_exec']),                     'price_exec': float(row['price_exec']),                     'missed_trade': 'override'})          print(f"  OVERRIDE {row['ticker']}  qty={int(row['quantity_exec']):+d}")

Handling exception trades to the TradeLog.

Missed and override trades are auditable in Airtable.

Now, we are defining position handlers.

Cell 8: position handlers

Four code paths cover every possible outcome for a matched trade:

This block of code is difficult to read. We left comments to illustrate the articulations.

Now, we will reconcile all the csv files and amend the portfolio and TradeLog.

Cell 9: reconcile

The orchestrator. Calls _split_trades once, writes the two exception categories, then routes each matched trade through a three-way decision: open new position, add to existing position, or close FIFO. Direction is determined by comparing the sign of quantity_exec with the sign of the existing position.

def reconcile(orders, executions, portfolio):      matched, missed, override = _split_trades(orders, executions)      _write_missed(missed)      _write_override(override)      port = portfolio.copy().reset_index(drop=True)      for _, row in matched.iterrows():          mask = (port['ticker']==row['ticker']) & (port['strategy']==row['strategy'])          if not mask.any():              port = _open_new(row, port)          elif (int(row['quantity_exec'])>0)==(int(port.loc[mask,'position'].values[0])>0):              port = _add_lot(row, port)          else:              port = _close_fifo(row, port)      return port.reindex(columns=PORTFOLIO_COLS)

Orchestrating the full reconciliation in a single function call.

One function, one call returns the updated portfolio df and writes to Airtable.

Now, let's run this function.

Cell 10: run

Calls reconcile with the three DataFrames, saves the updated portfolio to CSV, and displays the result. Replace the sample DataFrames with load_csv calls in production.

print('=== Reconciling ===')  updated_portfolio = reconcile(sample_orders, sample_executions, sample_portfolio)  print()  print('=== Updated portfolio ===')  display(updated_portfolio)  updated_portfolio.to_csv('sample_portfolio.csv', index=False)  print('Saved sample_portfolio.csv')

Executing the reconciliation on sample data.

In production, replace sample DataFrames with load_csv calls pointing at broker export files. It would also be judicious to make a timestamped copy of the sample_portfolio.csv to keep a record.

This concludes the TradeLog portion of the journal. Now, let's move to the psychological part.

Part B: psychology journal analysis

Part B runs weekly, or on demand. It loads all pre-market and post-market journal entries from Airtable, pairs morning, and evening rows by date, computes four derived fields (bulls_eye, score, streak, multiplier), patches the results back to Airtable, and displays a summary of the last ten trading days.

The five computed fields are:

Field

Formula

Meaning

bulls_eye

sign(perf_forecast × perf_actual)

+1 right call, −1 wrong call, 0 neutral or missing

Score

weighted sum of filled fields

completeness reward, max 120 pts

Streak

consecutive weekday entries

habit momentum; one missed weekday forgiven

Multiplier

1 + streak / 20

geometric compounding: day 20 = 2×, day 40 = 3×

final_score

score × multiplier

total gamified score

Table 10.5: Journal computed fields

The daily input requires no notebook. The morning form is opened on the phone before the open; the evening form after the close. The notebook only runs when we want to analyze the results.

Now, we define the function to load journal entries.

Cell 11: load_journal_session

The function defined below fetches all rows for one session type with automatic pagination. Airtable returns at most 100 records per page; the function follows the offset cursor until no offset is returned. When the table is empty it returns a DataFrame with a date column so the merge in Cell 13 never raises a KeyError.

def load_journal_session(session):      formula = f"{{session}}='{session}'"      records, offset = [], None      while True:          params = {'filterByFormula': formula,                    'sort[0][field]': 'date', 'sort[0][direction]': 'asc'}          if offset: params['offset'] = offset          r = requests.get(AT_J_URL, headers=AT_HEADERS, params=params)          r.raise_for_status()          data = r.json()          records += data.get('records', [])          offset   = data.get('offset')          if not offset: break      rows = [{'_id': rec['id'], **rec['fields']} for rec in records]      return pd.DataFrame(rows) if rows else pd.DataFrame(columns=['_id','date'])    pre_df  = load_journal_session('pre')  post_df = load_journal_session('post')  print(f'pre rows: {len(pre_df)}  |  post rows: {len(post_df)}')

Loading journal rows from Airtable.

Now, we will compute a few fields.

Cell 12: compute functions

Once the entries are loaded, we compute a few fields. The premise of the journal is to encourage what is working, not penalize failures. We will compute forecast directional accuracy, tally up score and compute streaks.

Now, let's look at the code.

def _bulls_eye(pf, pa):      if pd.isna(pf) or pd.isna(pa): return None      p = float(pf) * float(pa)      return 1 if p > 0 else (-1 if p < 0 else 0)    def _score(row):      s  = 20 if row.get('checklist_done')==1  else 0      s +=  5 if pd.notna(row.get('mindset_pre'))    else 0      s +=  5 if pd.notna(row.get('confidence_pre')) else 0      s +=  5 if pd.notna(row.get('perf_forecast'))  else 0      s +=  3 if str(row.get('bc_1') or '').strip()  else 0      s +=  2 if str(row.get('bc_2') or '').strip()  else 0      s +=  5 if pd.notna(row.get('mindset_post'))   else 0      s +=  5 if pd.notna(row.get('confidence_post'))else 0      s +=  5 if pd.notna(row.get('perf_actual'))    else 0      s +=  5 if str(row.get('what_went_well_1') or '').strip() else 0      s +=  5 if str(row.get('what_went_well_2') or '').strip() else 0      s +=  5 if str(row.get('what_went_well_3') or '').strip() else 0      s += 20 if str(row.get('tomorrows_kaizen') or '').strip() else 0      s += 10 if str(row.get('notes') or '').strip()            else 0      g  = sum(1 for k in ('gratitude_1','gratitude_2','gratitude_3')               if str(row.get(k) or '').strip())      return s + {0:0,1:6,2:12,3:20}[g]    def _add_streaks(df):      df = df.sort_values('date').copy()      streaks, n, prev = [], 0, None      for d in pd.to_datetime(df['date']):          if prev is None: n = 1          else:              gap = len(pd.bdate_range(prev + pd.Timedelta(days=1), d))              n   = n + 1 if gap <= 2 else 1          streaks.append(n); prev = d      df['streak']     = streaks      df['multiplier'] = (1 + df['streak']/20).round(2)      return df

Defining the three functions that transform raw journal fields into gamified performance metrics.

This creates three functions with no Airtable dependency.

Next, let's merge and compute the data.

Cell 13: merge and compute

An outer merge on date pairs each morning row with its evening counterpart. Days with only a pre or only a post entry are kept so nothing is silently dropped. The merged variable is initialized as an empty DataFrame before the guard block so that Cells 14 and 15 always find it defined.

merged = pd.DataFrame()    if not (pre_df.empty and post_df.empty):      merged = pd.merge(          pre_df.rename(columns={'_id': '_pre_id'}),          post_df.rename(columns={'_id': '_post_id'}),          on='date', how='outer', suffixes=('_pre','_post')      )      merged['bulls_eye']   = merged.apply(          lambda r: _bulls_eye(r.get('perf_forecast'), r.get('perf_actual')), axis=1)      merged['score']       = merged.apply(_score, axis=1)      merged                = _add_streaks(merged)      merged['final_score'] = (merged['score'] * merged['multiplier']).round(1)      print(f'Days merged: {len(merged)}')

Joining morning and evening rows by date and computing all derived metrics in one pass.

Now, let's patch the results into Airtable.

Cell 14: patch to Airtable

Patches the five computed fields onto the post row for each day. Days with no post row yet are skipped. The operation is idempotent: patching the same values twice has no side effects.

for _, row in merged.iterrows():      post_id = row.get('_post_id')      if pd.isna(post_id) if isinstance(post_id, float) else not post_id:          print(f'  SKIP  {row["date"]}  no post row yet')          continue      fields = {col: row[col] for col in                ('bulls_eye','score','streak','multiplier','final_score')                if row.get(col) is not None}      requests.patch(f'{AT_J_URL}/{post_id}',                     headers=AT_HEADERS, json={'fields': fields}).raise_for_status()

Writing the five computed fields back to each evening row in Airtable so they appear on the dashboard without rerunning the notebook.

All post rows in Airtable are updated with current scores. It is safe to re-run: patching the same values twice has no side effects.

Cell 15: summary

The last ten trading days, most recent first. Only columns present in the merged DataFrame are shown, so the cell runs cleanly even with partial data.

COLS = ['date','streak','multiplier','score','final_score',          'bulls_eye','mindset_pre','mindset_post','perf_forecast','perf_actual']  if not merged.empty:      avail = [c for c in COLS if c in merged.columns]      display(merged.sort_values('date', ascending=False)[avail].head(10))

Displaying a compact review of the most recent 10 trading days.

This generates a table showing streak, score, bulls_eye, mindset, and performance forecast versus actual for the last 10 days.

AI analysis of the psychology journal

The psychology journal generates structured data that an AI model can read with the patience and pattern-recognition capacity that most traders lack when reviewing their own notes. The prompts below are designed to be used with any frontier AI. Export the relevant journal rows from Airtable as CSV, paste them into the prompt, and submit.

The key principle is that the AI does not evaluate performance. It looks for patterns in behavior. A bad week in profit and loss (P&L) terms can reflect excellent psychology if execution was consistent. A strong week in P&L can mask deteriorating habits. The AI sees both and distinguishes between the attachment to the outcome or the process.

Weekly analysis

Run at the end of each week with the last five days of journal data. This prompt combines directional accuracy (bulls_eye) with reasoning quality (bc_1 and bc_2), distinguishing between traders who predict correctly by luck and those who understand the drivers.

You are a trading psychologist reviewing a systematic trader's journal. Here are the last five days of entries: [paste CSV]. Identify recurring patterns in mindset scores, confidence levels, and free-text notes. Note whether the pre-market checklist was completed consistently. Analyze the bulls_eye accuracy: on days where the trader called direction correctly (bulls_eye = +1), compare their bc_1 and bc_2 reasoning against the actual session driver. On days where they were wrong (bulls_eye = ‑1), identify whether the reasoning was structurally flawed or simply unlucky. Do not evaluate P&L. Focus on whether the trader is developing genuine understanding of performance drivers, not just directional guessing. End with one concrete suggestion for next week.

Kaizen: recurring themes and suggestions

This prompt reads the tomorrows_kaizen field across multiple days and identifies whether stated intentions are converting into improved behavior or simply recurring without resolution.

You are a performance coach. Here are 30 days of a trader's kaizen entries with dates: [paste tomorrows_kaizen column and dates]. Identify the three most common themes. For each theme, determine whether it recurs without evolution (an intention that is not converting into behavior change) or whether it evolves over time (genuine improvement in progress). Rank themes by urgency. For the most persistent unresolved theme, suggest one structural change: a rule, a checklist item, or a pre-market ritual that would address the root cause rather than the symptom.

What went well: strengths and reinforcement

Most journaling systems focus on weakness. This prompt analyzes the what_went_well fields to identify recurring strengths, which are equally important because they reveal the behaviors worth doubling down on.

You are a strengths coach reviewing a trader's journal. Here are 30 days of what_went_well entries: [paste what_went_well_1, what_went_well_2, what_went_well_3 columns with dates]. Identify the three most frequently recurring strengths or positive patterns. For each strength, suggest one way the trader could systematically reinforce or amplify it: a protocol, a habit, or a rule that locks in the behavior. Distinguish between strengths that appear consistently regardless of market conditions (structural) and those that only appear on good days (conditional). Do not comment on losses or weaknesses.

Monthly synthesis

The monthly synthesis reads the full journal for the month and produces a coaching report that covers both the analytics and psychology dimensions.

You are a trading psychologist writing a monthly review. Here is one month of journal data: [paste full month CSV]. Describe the trader's dominant emotional pattern: their most common mindset state, confidence level, and the recurring themes in their free-text notes. Identify the three sessions where psychology most likely affected trading outcomes (look for combinations of low confidence, low bulls_eye accuracy, and weak bc_1/bc_2 reasoning). Write two paragraphs: the first describes what is working and which strengths to reinforce; the second identifies the single most important pattern to address next month. Focus exclusively on execution psychology. Do not recommend changes to the trading strategy.

Streak recovery

When a streak breaks, this prompt diagnoses the cause and suggests a friction-reduction change.

A trader's journaling streak was broken after [N] consecutive days. Here are the last 7 days of entries including the gap: [paste CSV]. Identify what was different in the days before the gap: lower mindset scores, shorter entries, unusual notes. Suggest one concrete friction-reduction change that would have made it easier to complete the journal on the missed day. Keep the response under 200 words.

Next, let's build the TradeLog analytics.

TradeLog analytics

The TradeLog contains the raw material for a rigorous post-trade analytics practice. Three functions answer the questions that P&L alone cannot: how badly did individual trades move against us during the holding period? How often and how deeply does the equity curve dip into drawdown? How many consecutive losses do the strategies produce on average, and what is the highest consecutive number of losses?

These functions are intentionally minimal. Each does one thing. None requires a live Airtable connection: they operate on a pandas DataFrame of closed trades. Load the TradeLog as a CSV or fetch records from Airtable and pass them in.

This minimalism obeys a simple philosophy: focus on the losses and the profits will take care of themselves. This is where we need to take a long hard look at our trading history. The purpose is to analyze losses and look for continuous improvements. We will look at Maximum adverse excursion (MAE), drawdowns, and consecutive losses.

Maximum adverse excursion

MAE measures the worst adverse price move from entry to exit, expressed as a percentage of the entry price. For a short position, adverse means price rising; for a long position, adverse means price falling. MAE is computed over the full holding period using daily close prices, not intraday data, and is used to refine stop loss placement. If MAE far exceeds the average stop loss, then stop losses can be tightened. Conversely, if the loss rate is too high, loosen and observe the MAE.

MAE is particularly important for left skewed strategies. In fact, the ratio Maximum Favorable Execution MFE/MAAE is the most robust indicator for the reclassification of strategies. Trend followers have a MFE/MAE > 1. Mean reversion strategies have a MFE/MAE < 1.

The function takes two DataFrames: the closed trades, and a daily close price Series indexed by date. For each trade it extracts the relevant price window and identifies the worst daily close relative to the entry price.

def mae(trades, prices):      """MAE as % of entry: worst daily close vs entry over holding period.      trades: DataFrame with entry_date, exit_date, price_exec, quantity_exec.      prices: Series of daily close prices indexed by date (one ticker at a time).      Returns: pd.Series of MAE values (percentage points, >= 0).      """      out = []      for _, t in trades.iterrows():          p = prices.loc[t['entry_date']:t['exit_date']]          if t['quantity_exec'] < 0:   # short: adverse = price up              out.append(max(0, (p.max() - t['price_exec']) / t['price_exec'] * 100))          else:                          # long: adverse = price down              out.append(max(0, (t['price_exec'] - p.min()) / t['price_exec'] * 100))      return pd.Series(out, index=trades.index)

Computing MAE.

This generates a pd.Series of MAE values in percentage points (≥ 0).

Next, let's analyze the drawdown.

Drawdown analysis

Drawdown analysis operates on the cumulative net P&L series across closed trades in chronological order. A drawdown period begins when cumulative P&L falls below its previous peak and ends when it recovers.

Investors claim they look for returns, and yet they react to drawdowns. Investors care about drawdowns in three specific ways:

  1. Magnitude: Never test the stomach of your investors.
  2. Frequency: Never test the nerves of your investors.
  3. Duration: Never test the patience of your investors.

The function returns maximum depth, average depth, maximum duration in trades, average duration, total count of drawdown periods, and frequency as a ratio of drawdown periods to total trades.

def drawdown_stats(pnl):      """Drawdown statistics from a net P&L series.      pnl: array-like of net_pnl values in chronological order.      Returns dict: max_dd, avg_dd, max_dur, avg_dur, count, freq.      """      cum = pd.Series(pnl).cumsum()      dd  = (cum - cum.cummax()) / cum.cummax().clip(lower=1e-9)      in_dd, periods, s = dd < 0, [], None      for i, v in enumerate(in_dd):          if v and s is None: s = i          elif not v and s is not None: periods.append((s, i-1)); s = None      if s is not None: periods.append((s, len(dd)-1))      if not periods:          return dict(max_dd=0, avg_dd=0, max_dur=0, avg_dur=0, count=0, freq=0)      d = [dd.iloc[s:e+1].min() for s,e in periods]      n = [e-s+1                for s,e in periods]      return dict(max_dd=round(min(d),4), avg_dd=round(sum(d)/len(d),4),                  max_dur=max(n), avg_dur=round(sum(n)/len(n),1),                  count=len(periods), freq=round(len(periods)/len(cum),4))

Computing drawdown statistics: magnitude, duration, and frequency from a net P&L series.

This generates a dict with six metrics: max_dd, avg_dd, max_dur, avg_dur, count, freq. Depth values are negative ratios (e.g. −0.18 = 18 % drawdown). Use freq to understand how much of the time you are underwater; use max_dur to plan how long you must psychologically tolerate a losing streak.

Every loss takes its toll on mental health. This is why it is much more important to focus on consecutive losses than wins.

Consecutive loss analysis

The consecutive loss count shows the maximum number of losing trades a strategy has produced in a row, and the average length of losing runs. This number matters for position sizing and for psychological preparation. Reduce size in a losing streak to preserve emotional and financial capital.

def consecutive_losses(results):      """Max and average consecutive loss runs.      results: array-like of 'win', 'loss', or 'breakeven' strings.      Returns dict: max_run, avg_run.      """      runs, c = [], 0      for r in results:          if r == 'loss': c += 1          elif c: runs.append(c); c = 0      if c: runs.append(c)      return dict(max_run=max(runs, default=0),                  avg_run=round(sum(runs)/len(runs), 1) if runs else 0.0)

Identifying the longest and average consecutive loss streaks in a trade log.

A dict with max_run (worst case) and avg_run (typical streak length). Use max_run to stress-test drawdown tolerance and position sizing.

Loading the TradeLog records from Airtable

Let's load the trading history from Airtable.

def load_tradelog():      """Load all TradeLog records from Airtable, return as DataFrame."""      records, offset = [], None      while True:          params = {'sort[0][field]': 'entry_date', 'sort[0][direction]': 'asc'}          if offset:              params['offset'] = offset          r = requests.get(AT_URL, headers=AT_HEADERS, params=params)          r.raise_for_status()          data     = r.json()          records += data.get('records', [])          offset   = data.get('offset')          if not offset:              break      rows = [{'_id': rec['id'], **rec['fields']} for rec in records]      return pd.DataFrame(rows) if rows else pd.DataFrame(          columns=['_id', 'entry_date', 'exit_date', 'net_pnl', 'trade_result'])    tradelog_df = load_tradelog()  closed_count = tradelog_df['exit_date'].notna().sum() if 'exit_date' in tradelog_df else 0  print(f'Loaded {len(tradelog_df)} TradeLog records ({closed_count} closed)')

Loading the full TradeLog from Airtable into a DataFrame.

This generates tradelog_df: all open and closed lots in chronological order. The analytics cell filters to closed trades via dropna(subset=['exit_date']).

Now, let's put everything together.

Using the three functions together

Load the closed trades, filter to rows with a non-empty exit_date, sort by exit_date, and pass the relevant columns to each function. A typical weekly review would look like this:

# Load and filter closed trades  closed = tradelog_df.dropna(subset=['exit_date']).sort_values('exit_date')    # MAE (requires a daily close price Series for the relevant ticker)  mae_series = mae(closed, daily_prices)  print(f'Avg MAE: {mae_series.mean():.1f}pp   Max MAE: {mae_series.max():.1f}pp')    # Drawdown  dd = drawdown_stats(closed['net_pnl'])  print(f'Max DD: {dd["max_dd"]:.1%}  Avg DD: {dd["avg_dd"]:.1%}  '        f'Max duration: {dd["max_dur"]} trades  Frequency: {dd["freq"]:.1%}')    # Consecutive losses  cl = consecutive_losses(closed['trade_result'])  print(f'Max losing run: {cl["max_run"]}  Avg losing run: {cl["avg_run"]}')

As a personal parting note, my sincere hope is that some of you will switch your addiction from social media or gaming to journaling. This is the path to trading mastery.

Summary

This chapter started from a single premise: debugging the programmer is harder than debugging the code, and it requires its own systematic tooling.

You now have that tooling. The Airtable base holds two tables: the TradeLog captures every trade, missed trade, and override in a FIFO-aware structure that distinguishes between intent and execution; the journal table captures the pre-market and post-market mental state through two mobile forms that together require seven minutes of daily effort.

Part A automates the reconciliation of those two realities. A single outer merge classifies every order-versus-execution pair into matched, missed, or override, routes matched trades through four position handlers that correctly manage pyramids, partial exits, and reversals, and writes the full audit trail to Airtable without manual intervention.

Part B automates the psychology loop. A gamified scoring system rewards completeness over perfection. A streak multiplier compounding at 1 + streak/20 creates a geometric incentive to maintain daily journaling. Bulls_eye accuracy tracks the directional judgement against the stated reasoning, separating genuine understanding from lucky guessing.

The AI prompts layer intelligent pattern recognition on top of this data, transforming the journal from a private diary into a coaching conversation. Weekly analysis spots emotional patterns and calibration gaps. The Kaizen prompt identifies recurring intentions that are not converting into behavior change. The what-went-well prompt surfaces recurring strengths worth reinforcing.

The three analytics functions complete the quantitative picture. MAE, computed over the full holding period from daily closes, tells whether stop placements are calibrated to how trades actually behave before they resolve. Drawdown statistics tell how deep and how frequent equity drawdowns are. Consecutive loss analysis tells the psychological toll it will claim before it pays off.

Together, these tools answer the only question that matters once a trading edge has been established: are we adhering to the discipline consistently, and if not, when and why not?

From the bottom of this unprofitable organ called my heart, thank you for taking the time and dedication to complete this book. I hope the ideas and insights you have discovered here will continue to inspire you, challenge you, and guide you forward. I hope the students will go further than the teacher.

Get this book's PDF copy, code bundle, and more

Scan the QR code (or go to ). Search for this book by name, confirm the edition, and then follow the steps on the page.

Image

Image

Note: Have your invoice handy. Purchases made directly from the Packt website don't require an invoice.

Назад: Chapter 9: Asset Allocation
Дальше: Chapter 11: Unlock Access to the Code Bundle and the PDF Version