Книга: Algorithmic Short Selling With Python
Назад: Part 2: The Outer Game: The Trading Edge
Дальше: Chapter 5: The Trading Edge Is a Number, and Here Is the Formula

4

Regime Definition

During the Napoleonic wars, field surgeons with limited resources had to make quick decisions as to who would need surgery, who could survive without it, and the unfortunate ones for whom nothing could be done. Triage was born out of necessity to allocate limited time and resources in the most efficient and humane way possible. In the stock market, regime is another word for triage. Some are bullish, some are bearish, and some are inconclusive.

Markets tend to stay wrong a lot longer than investors tend to stick with you. Segregating stocks into different regime buckets—triaging them—before performing in-depth analysis is an efficient allocation of resources. The objective of this initial triage is not to predict where stocks could, would, or should be headed, but to practice the long-lost art of actively listening to what the market has to say.

Some market participants like to spend time and resources building bearish cases for bullish stocks that insolently defy the gravity of reason. This is not efficient for two reasons. First, they expect reversion to the mean. On the long side, they trade trends and ride outperformers, expecting them to continue to do well. Meanwhile, on the short side, they trade mean reversion, expecting expensive stocks to choke on humble pie and come back down to cheap prices again.

As a different approach, establishing a market regime is something that could really help fundamental short-sellers. They often show up too early. They place their bets long before the broader market starts to discount the information. The difference between a short selling guru and the dreaded tap on the shoulder is 6 months. Short internet stocks in 1999, and you'll be teaching math to bored university students in 2000. Short the same stocks as early as late January 2000, and a new short selling star is born.

We will go through classic regime definition methodologies. We will define breakouts, turtle traders, and moving average crossovers. We will then calculate fractals and use them to define higher highs and lows, as well as floor and ceiling methodologies. We will plot each regime definition to highlight the differences between each method. We will process the entire S&P 500 and display the sector rotation we talked about in the previous chapter. Finally, we will calculate fractals and regimes across various timeframes, from 5 minutes to daily bars.

In the following sections, we will look at various regime definition methods before blending them:

Technical requirements

You can access color versions of all images in this chapter via the following link: .

You can also retrieve the source code for this chapter via the book's GitHub repository. To access the repository link, follow the steps in the Download the example code files section in the Preface.

Importing libraries

For this chapter and the rest of the book, we will be working with the pandas, numpy, yfinance, and matplotlib libraries. We will also be working with Mplfinance, a charting library built on matplotlib.

import pandas as pdimport numpy as npimport datetime as dtimport timeimport pathlibimport arcticdb as adbfrom arcticdb import LibraryOptionsimport requests                from io import StringIO  import yfinance as yf%matplotlib inlineimport matplotlib.pyplot as pltimport mplfinance as mpf

If you do not have any of those libraries installed, you know the drill with pip or conda install:

pip install –upgrade library_name

Utilities functions

We will pick up where we left off in . First, we will import the necessary utilities functions.

Here is a list of the utilities functions we will be using, followed by their respective source code:

In a nutshell, we download data in batches of 51 securities at a time. We start from January 2015. We will benchmark the series to the S&P in USD ('local'). We have 3 durations: 20, 50, and 100 days. We will process the relative series first, and the absolute ones second. We will calculate scores for the absolute and relative series, as well as a composite of both.

Next, let's build the currency (ccy_df) and benchmark (bm_df) dataframes.

dict_bm =  {    '^GSPC':'SP500','^DJI':'DOW30','^IXIC':'Nasdaq100','^NYA':'NYSE ',    '^XAX':'Nyse_Amex','^RUT':'Russell2000','^VIX':'VIX',    '^BUK100P':'CBOEUK100','^FTSE':'FTSE100','^GDAXI':'DAX','^FCHI':'CAC40',    '^STOXX50E':'EuroStocks50','^N100':'Euronext100','^BFX':'BEL20',    '^N225':'Nikkei225','^HSI':'HangSeng','000001.SS':'SSE','399001.SZ':'Shenzhen'}dict_ccy = {    'EURUSD=X':'EURUSD','JPY=X':'USDJPY','GBPUSD=X':'GBPUSD',    'AUDUSD=X':'AUDUSD', 'NZDUSD=X':'NZDUSD', 'USDCAD=X': 'USDCAD',    'EURJPY=X':'EURJPY','GBPJPY=X':'GBPJPY','EURGBP=X':'EURGB,    'EURCAD=X':'EURCAD','EURSEK=X':'EURSEK','EURCHF=X':'EURCHF',    'CNY=X':'USDCNY','HKD=X':'USDHKD','SGD=X':'USDSGD','INR=X':'USDINR',    'KRW=X' : 'USDKRW', 'TWD=X': 'USDTWD',}ccy_df = df_from_dict(batch_size,dict_ccy,start,end)ccy_df['local'] = 1bm_df = df_from_dict(batch_size,dict_bm,start,end).round(2)bm_df['absolute'] = 1print(bm_df.shape, ccy_df.shape)

Next, let's download the major automakers' historical OHLCV data. In order of appearance, we have Toyota (7203.T), Nissan (7201.T), BYD (1211.HK), Hyundai Motors (005380.KS), Volkswagen (VOW3.DE), Renault (RNO.PA), Ford (F), Tesla (TSLA), and General Motors (GM).

bm = 'SP500'tickers_fx_dict = {    '7203.T': 'USDJPY', '7201.T': 'USDJPY',    '1211.HK':'USDHKD','005380.KS':'USDKRW','VOW3.DE' :'EURUSD',     'RNO.PA':'EURUSD','F':'local' ,'TSLA':'local' ,'GM':'local'  }tickers_list = list(tickers_fx_dict.keys())multiIndex_raw_data = batch_df(tickers_list,batch_size, start, end,show_batch = False).round(2)multiIndex_raw_data.tail()

We have created a multiIndex_raw_datadf that contains the OHLC prices for all the tickers named above.

In order to visually illustrate how various regimes look in practice; we will pick one stock from the list and visualize each regime. And the winner is the Rolls Royce (pun intended) of short-sellers, Nissan (7201.T).

ticker = '7201.T'raw_df = yf_droplevel(multiIndex_raw_data,ticker)_o,_h,_l,_c = rohlc(raw_df,relative = False)df = rel_fx(raw_df,_o,_h,_l,_c, bm_df, bm, ccy_df, tickers_fx_dict[ticker], start, end,rebase=True, mult=1)df.shape

Now that the supporting code and functions are out of the way, let's proceed to the next stage: regime definition, starting with range breakout.

Regime definition: defining methodologies

We will proceed to define regime using various methodologies. We will start with the oldest methodology, breakout.

Breakout/breakdown method

"Kites rise highest against the wind—not with it."

– John Neal, posthumously attributed to Sir Winston Churchill

Breakout is the oldest and simplest trend-following method. It works for both bull and bear markets. If the price makes a new high over x number of periods, the regime is bullish. If the price makes a fresh low over x number of periods, the regime is bearish. This method is computationally easy to implement.

Popular durations are 252 trading days (one business year), 100 trading days, and 50 trading days. The following code is a simple rendition of this regime methodology:

def regime_breakout(df,_h,_l,n):    hl =  np.where(df[_h] == df[_h].rolling(n).max(),1,                   np.where(df[_l] == df[_l].rolling(n).min(), -1, np.nan))    roll_hl = pd.Series(index= df.index, data= hl).ffill()    return roll_hl

The way this code functions is simple:

  1. If the high is the highest of x periods, then hl is 1.
  2. Otherwise, if the low is the lowest of x periods, then hl is ‑1.
  3. If neither of these conditions is true, hl is N/A.
  4. We want to propagate the latest values forward along the missing values using the ffill() method. First, we convert the numpy array to a pandas series.
  5. Then, we fill missing values using the ffill() forward fill method.

In , we displayed the currency-adjusted relative series for various auto manufacturers around the world. In this chapter, we will take this example one step further. We will display various regime definition methodologies for both absolute and relative series.

One picture is worth a thousand words. We will therefore illustrate each regime methodology with its shadows. We will repeat the same block of code with different methodologies and plot the result. We will publish the code once.

fields_list = [] ; fields_list_rel = [] ; subplots = [] ; pane = 1mpf_relative(df[start_mpf:], rel=True, sma_list= sma_list, pane=1, color= 'k')for rel in rel_list:    _o,_h,_l,_c = rohlc(df,rel)       col_name = col_name_abs_rel(rel, name_abs = 'abs ', name_rel = 'rel ')      for t, slt in enumerate(sorted(bo_list)):        bo_field = f'{col_name}BO{slt}'        df[bo_field] = regime_breakout(df,_h,_l,slt)                _min = df[start_mpf:][_l].rolling(slt).min()        _max = df[start_mpf:][_h].rolling(slt).max()        mpf_fill(df[start_mpf:], _min, _max, color_bull, color_bear, alfa/(t+1), pane, field= bo_field)        fields_list.append(bo_field)     if rel:        fields_list_rel += fields_list         pane = 0 fields_list_abs = list(set(fields_list) - set(fields_list_rel)) df['score_abs'] = round(df[fields_list_abs].mean(axis=1),2).valuesdf['score_rel'] = round(df[fields_list_rel].mean(axis=1),2).valuesdf['score'] = round(df[fields_list].mean(axis=1),2).valuesmpf_score(df[start_mpf:], score_list, alfa, color_bull, color_bear, pane= 2, vlabel = 'score')mpf.plot(df[start_mpf:], type= 'ohlc', style= 'charles', figsize= (16,4), title =  f'{ticker}  {fields_list}', mav = tuple(sorted(sma_list)),             addplot= subplots, volume= True, volume_panel = 3, panel_ratios = (2,2,0.5,0.5))

This block of code will print every regime methodology for every stock. This will rapidly become repetitive without much marginal contribution. We will go through the process once:

  1. We instantiate lists and variables.
  2. We add the relative subplot.
  3. We loop the relative list, starting with relative, then absolute.
  4. We calculate the regime method, regime_breakout.
  5. We loop through the breakout list bo_list [20, 50, 100]. The longer the duration, the lighter the shade of color_bull or color_bear.
  6. We build a list of fields or columns to calculate the relative score list.
  7. We calculate the absolute list by subtracting the relative fields from the overall list.
  8. We calculate scores for the absolute, relative, and composite series.
  9. We add the score subplot.
  10. We plot the chart.

And this is the result:

ChA screenshot of a graph

Figure 4.1: Nissan breakout regime definition

Nissan fits the definition of a structural short. This is a low-volatility, long-way-down. Very few shorts are like this example.

This range of breakout strategy works wonders when the price breaks out after consolidation or a sideways market. Sideways markets are the interim periods between upward or downward trends, when the old regime agonizes, and the new has yet to flourish. In a sideways, undulating market, price oscillates in a range.

Bulls fight bears in a Gilgamesh epic battle. When prices break out from the upper or lower bound, this signals one side has thrown in the towel. Pent-up energy is released. The price moves effortlessly along the line of least resistance. The main drawback of this method is its built-in lag, which comes from the lookback period. In financial revisionist jargon, the waiting period is called confirmation. Jittery market participants rarely have the serene, enlightened equanimity to twiddle their thumbs for 50 days, 100 days, or even a year to finally find some resolution. Time is money. Stocks late on their rent should either be reduced or kicked out. Market participants who use this method with a long duration may want to re-introduce a partial time exit into their strategy. For example, halve size with positions that have failed to make new highs/lows after half the duration. Are resources better deployed on a slow burning low volatility or explosive market?

The main advantages of this method are computational simplicity and stability. The major drawbacks are its inherent lag and the discomfort of giving back large amounts of profits. This leads us to the next iteration: the asymmetrical range breakout strategy.

Further refinements to the breakout regime definition method include dissociated periods for entries and exits. For example, the legendary Chicago Turtle Traders entered on 50-day highs and closed on 20-day lows. This asymmetrical duration is a subset of the breakout methodology.

Turtle traders

The turtle strategy is a rudimentary script inspired by the legendary Turtle Traders. It is composed of two range breakout regimes. The slower duration is used for entries. The faster duration is used for exits. This asymmetrical duration for entry and exit relies on a time-honored principle: be prudent and deliberate to confirm trends, but quick and decisive to cut losses and protect profits. This last regime will be renamed simplified Turtle from now on. It is a rudimentary script, yet it still works.

We will be recycling this basic strategy throughout this book to illustrate examples, purely for educational purposes. However, do not do this at home—it is realistic enough for educational purposes, but too simplistic to be deployed in real-money production.

def turtle_trader(df, _h, _l, st, lt):    '''    _lt: Long/Short direction    _st: trailing stop loss    '''    _lt = regime_breakout(df,_h,_l,lt)    _st = regime_breakout(df,_h,_l,st)    turtle = pd. Series(index= df.index, data = np.where(_lt == 1,np.where(_st == 1,1,0), np.where(_lt == -1, np.where(_st ==-1,-1,0),0)))    return turtle

This script builds on the regime_breakout function. When the _lt does not match the _st, go neutral.

The longer duration gives the direction: long or short. The shorter duration is the stop loss. We will discuss stop losses in later chapters. The shorter duration protects profits by narrowing the range. The flipside of this is an elevated trading frequency, higher transaction costs, and turnover. This set of parameters for the Turtle strategy does not work well in sideways, choppy markets. We will skip the code and directly plot the corresponding graphs. This produces the following chart:

A graph of a stock market AI-generated content may be incorrect.

Figure 4.2: Nissan Simplified Turtle

This asymmetrical duration enables traders to capture small profits in nimble markets. The drawback of the turtle strategy is the increase in trading frequency. Even though the general direction is correct, you may get stopped a few times in volatile markets. This will increase transaction costs and slippage and reduce the efficacy of the strategy.

Next, let's go over the classic moving average crossover strategy.

Moving average crossover

Moving average crossover is another classic regime definition method. This method is so simple and prevalent that even the most hardcore fundamental analysts who claim never to look at charts still like to have a 200-day simple moving average. This method is also computationally easy. There may be further refinements as to the type of moving averages, from simple to exponential, weighted, triangular, or adaptive. Yet, the principle remains the same. When the faster moving average is above the slower one, the regime is bullish. When it is below the slower one, the regime is bearish. The following code shows how to calculate the regime with two moving averages using simple and exponential moving averages (SMA and EMA respectively):

def regime_sma(df,_c,st,lt):    sma_lt = df[_c].rolling(lt).mean()    sma_st = df[_c].rolling(st).mean()    rg_sma = np.sign(sma_st - sma_lt)    return rg_smadef regime_ema(df,_c,st,lt):    ema_st = df[_c].ewm(span=st,min_periods = st).mean()    ema_lt = df[_c].ewm(span=lt,min_periods = lt).mean()    rg_ema = np.sign(ema_st - ema_lt)    return rg_ema

This regime_sma() function is as simple as it gets. Regime is the sign difference between the short-term and long-term moving averages. Next, we will plot the regime_sma().

We recycle the mpf_fill function. Instead of calculating a minimum and maximum, we repeat the sma function.

A graph of a stock market AI-generated content may be incorrect.

Figure 4.3: Nissan simplified turtle

The lighter shade is the envelope between the short- and long-term moving averages. The darker shade is the space between the price and the shorter duration. Here, we can also observe the bull, _loss, and bear_loss. The longer moving average has a lighter shade.

Next, we will calculate fractals.

Fractals

"Talk is cheap. Show me the code."

– Linus Torvalds

Fractals are inspired by the seminal work of the Belgian mathematician Benoit Mandelbrot. The concept makes intuitive sense. A fractal low is a low surrounded by two higher lows. A fractal high is a high surrounded by two lower highs. In technical analyst creole, fractals are usually referred to as swings. For the rest of the discussion, think of fractals as swing highs and swing lows. Swings alternate: a fractal high follows a fractal low.

Here is a visual example of a succession of fractals: low, high, and low. Let's take a look.

Figure 4.4: GM fractal low

Figure 4.4: GM fractal low

The first fractal low level 1, symbolized by an upward triangle marker, is printed on the second bar from the left. The low of this bar is surrounded by two higher lows. The third bar from the left prints a swing low level 1, symbolized by a downward triangle marker. The high of bar 3 is higher than the highs on bars 2 and 4. The fourth bar prints another fractal low level 1, symbolized by an upward triangle marker. Bars 3 and 5 print higher lows than bar 4. So, bar 4 qualifies as a fractal low level 1. Let's recapitulate: bars 2 and 4 are fractal lows level 1. Bar 3 is a fractal high level 1. Level 1 stands for first level of abstraction. The source data is HLC price series.

The second fractal low on bar 4 is higher than the first one on bar 2. We have a fractal high level 1 between those fractal lows level 1. Since the bar preceding the left fractal also closes higher, this means the first fractal low level 1 on bar 2 also qualifies as level 2. We have a lower low surrounded by two higher lows. Level 2 stands for the second level of abstraction. The source data is fractals level 1 series. Level 3 will use level 2 data, and so on and so forth.

We have a rapid succession of fractal lows and highs. Note that the fractals are neither at the low nor at the high of their bar. Markets are noisy. Markets sometimes print a local low and a local high on the same day. However, the market cannot simultaneously print a fractal high and a fractal low. There has to be a succession of fractal highs and lows. We must determine which local high or low will become a fractal. There are essentially two ways to solve this simultaneous high/low occurrence. This boils down to which price series we choose to use to calculate fractals.

Either we choose a single point like the "Close" price, or we calculate the average of HLC price. We tried both. It quickly became apparent that the highest/lowest "Close" did not coincide with the true local "High/Low". The fractal based on the "Close" price would be a few bars away from the actual fractal. Therefore, by convention, we base our calculation on the average of the "High", "Low", "Close". Markets may err on the way up and down during a session, but where it chooses to end gives us a clue as to how things are going next.

Fractals are recursive, complex, two-dimensional series. In execution trader English, this means we calculate the current numbers from the previous series:

Fractals have three major benefits:

  1. Fractals work across timeframes. We can use the same algorithm to calculate fractals on the 1-minute bar as well as on weekly bars.
  2. Fractals work across asset classes. Fractals originally came from a subset of geometry called topography. They apply to virtually anything in nature. As a result, they apply to any asset class. Whether you analyze treasuries, stocks, cryptos, electricity load on the electric grid, or even real estate prices in Timbuktu, fractals work just the same way.
  3. Fractals do not require parameterization. Virtually, every indicator requires a parameter. For example, the default RSI setting is 14 periods. A simple moving average might use 20, 50, or 200 days, for instance. Those parameters do not translate well across time frames or even markets. Fractals do not need parameters.

Fractals may not need parameters, but they return levels of abstraction. For example, level 1 is derived from the price series. Those series are still very noisy. Level 2 series have filtered the noise. Level 3 gives inflections in trends, etc. In practical terms, swings level 1 are based on prices. Swings level 2 are based on swings level 1. Level 2 daily swings are predominantly used in swing trading. Swings level 3 are the bullish or bearish trends used in trend following trading.

Bottom line: In theory, we could potentially trace the inception of a bear market back to a specific 1-minute bar that triggered an avalanche. Hold that thought…

This method is a departure from the first edition of this book. This methodology is much faster, more accurate, and more computationally friendly.

We break it down into four functions:

  1. def avg_px(df, _h, _l, _c): Calculates the average price using HLC bars. We prepare the data into a single series that will be used in the fractals_df.
  2. def fractal(px, lvl): This function identifies fractals at multiple levels.
  3. def fractals_df(df, col, col_name): This function calculates fractal swing Highs and Lows at multiple levels of abstraction.
  4. def fractals_dates(df, col, col_name): This function calculates the dates when fractals at each level are discovered.

The first function calculates the average price from OLH. The second function identifies fractals.

def avg_px(df, _h, _l, _c):    return df[[_h,_l,_c]].mean(axis=1)def fractal(px, lvl):    max_lvl = np.minimum(2,lvl)    fractal = px[(px<= px.shift(-1))&(px< px.shift(+1)) &                 (px<=px.shift(-max_lvl))&(px< px.shift(+ max_lvl))]    return fractal

Next, let's go into more detail with the fractals_df function:

def fractals_df(df, col, col_name):    ffill_list = []    hilo = df[col].copy()    n = 1    temp_cols = {}  # Dictionary to store new columns    while len(hilo) >= 3:        lows = fractal(hilo, n)        if not lows.empty:            last_low = lows.iloc[-1:].copy()        else:            last_low = pd.Series()                 highs = -fractal(-hilo, n)        if not highs.empty:            last_high = highs.iloc[-1:].copy()        else:            last_high = pd.Series()                 hilo = lows.add(highs, fill_value=0).dropna()        level_low_col = f'{col_name}Lo{n}'        level_high_col = f'{col_name}Hi{n}'        hilo_col = f'{col_name}HiLo{n}'             if (not lows.empty) and (not highs.empty):            if (not last_low.empty) and (not last_high.empty) and (last_low.index[-1] > last_high.index[-1]):                lows = pd.concat([fractal(hilo, 1), last_low])                highs = -fractal(-hilo, 1)            elif (not last_low.empty) and (not last_high.empty) and (last_low.index[-1] < last_high.index[-1]):                lows = fractal(hilo, 1)                highs = pd.concat([-fractal(-hilo, 1), last_high])                         hilo = lows.add(highs, fill_value=0).dropna()            temp_cols[level_low_col] = lows            temp_cols[level_high_col] = highs            temp_cols[hilo_col] = hilo                 elif not lows.empty:            temp_cols[level_low_col] = lows            temp_cols[hilo_col] = lows                 elif not highs.empty:            temp_cols[level_high_col] = highs            temp_cols[hilo_col] = highs                 ffill_list.append(hilo_col)        n += 1             if hilo.empty:            break         # Concatenate all columns in temp_cols with df in one operation    if temp_cols:  # Check if temp_cols is not empty        temp_df = pd.DataFrame(temp_cols)        df = pd.concat([df, temp_df], axis=1)        df.dropna(how='all', axis=1, inplace=True)        df = df.loc[:, ~df.columns.duplicated(keep='last')]             list_ffill = [col for col in ffill_list if col in df.columns]        if list_ffill:  # propagate HiLo values forward            df[list_ffill] = df[list_ffill].ffill()             return df

This was a relatively long function. However, considering we solved Benoit Mandelbrot's conjecture in fewer than 100 lines, this is not bad.

  1. Create a hilo series from a specific column in the main df.
  2. Loop through as long as the hilo_df is longer than 3. Separate lows and highs by assigning a negative value to the series.
  3. Calculate last_low and last_high. If last_low happens later than last_high, concatenate last_low with hilo; else concatenate last_high with hilo.
  4. Update the temp_cols dictionary with lows, highs, and hilo.
  5. Append the ffill_list with hilo_col.
  6. Build a temp_df from the temp_cols.
  7. Concatenate the main df with temp_df.
  8. Remove DropNA and duplicate columns, saving the last one.
  9. Propagate the hilo values in the list_ffill using the .ffill() method.

This fractals_df() function finds historical swings. Next, let's try to identify when those fractals happen. If we want to find when a level 4 fractal happens, we need to identify the price bar that triggered a level 1, 2, and 3 fractal. There are essentially two ways to find those bars. The hard way is to loop through the dataframe and recalculate bar after bar.

The other way is to slice and reduce the dataframe. The following fractals_dates() function may be short, but it packs a serious punch.

def fractals_dates(df, col, col_name):    """    Identifies when fractal patterns are discovered and records:    1. The fractal value at discovery time (_fc)    2. The price at the next data point (_chg)    """    # Initialize a dictionary to hold temporary columns    temp_cols_dict = {}    for hl in ['Hi', 'Lo']:                   if (hl == 'Lo'):            high_cols_df  = pd.DataFrame.from_dict(temp_cols_dict)          hilos = df.copy()        n = 1 ; s = 1                     while f'{col_name}{hl}{n}' in hilos.columns:                     hilos = hilos[ (pd.notnull(hilos[f'{col_name}{hl}{n}'])) | (pd.notnull(hilos[f'{col_name}{hl}{n}'].shift(s))) ]                     temp_cols_dict.update({f'{col_name}HiLo{n}_chg': hilos.loc[ (pd.isnull(hilos[f'{col_name}{hl}{n}'])), col],                    f'{col_name}HiLo{n}_fc': hilos.loc[ (pd.isnull(hilos[f'{col_name}{hl}{n}'])), f'{col_name}HiLo{n}'] })               n +=1 ; s = 3    low_cols_df = pd.DataFrame.from_dict(temp_cols_dict)          hilo_cols_df = high_cols_df.add(low_cols_df, fill_value =0)    df = pd.concat([df, hilo_cols_df], axis=1)    df = df.loc[:, ~df.columns.duplicated(keep='last')]    df[hilo_cols_df.columns] = df[hilo_cols_df.columns].ffill()    return df

This function identifies the bar at which a higher-level fractal is detected. In theory, it can identify the 1-minute bar at which a daily bar market turned bullish or bearish. This is a bit like a Matryoshka, that is, a Russian nested doll. The df is reduced from level 1 to 2, 2 to 3, and so on. The logic may be intuitive, yet the computation is anything but trivial. So, let's go through the logic first, then we will go to the code implementation:

  1. A fractal 1 requires 3 prices: 1 peak or trough price surrounded by 2 lower or higher prices.
  2. A fractal 2 requires 3 fractals 1: 1 peak or trough fractal 1 surrounded by 2 lower or higher fractals 1.
  3. A fractal 3 requires 3 fractals 2: 1 peak or trough fractal 2 surrounded by 2 lower or higher fractals 2.
  4. A fractal 4 requires 3 fractals 3: 1 peak or trough fractal 3 surrounded by 2 lower or higher fractals 3, and so on.

Our job is to find the bar that preceded the fractal 1 that ended up at higher levels. So, for a level 5 to be detected, we need all the lower levels to be discovered, all the way down to the price action that triggered the level 1. This function identifies the bar after the peak/trough HLC series that will trigger a fractal 1. This will produce a fractal 2, which will generate a fractal 3, and so on. It is a gradual process of elimination of all the intermediate fractals leading to higher fractals. This is why level 3, 4, and 5 fractals are discovered months later. The logic is relatively simple. Now, let's go through the code:

  1. We loop through ['Hi', 'Lo']. We create a hilosdf by copying the main df.
  2. Once we have iterated over the 'Hi', we generate a dataframe from the dictionary temp_cols_dict.
  3. We use a while loop to iterate as long as 'Hi{n}' are found in the hilos dataframe.
  4. We slice the data to the column. We only keep the columns where:
    1. there is data at level {n}, and
    2. the price bar that triggered the fractal. At level 1, this is the following bar, hence shift(s) with s =1. For every subsequent level, s = 3.
  5. When rel == "Lo", we create another df from a dictionary.
  6. We create hilo_cols_df by adding both low_cols_df and high_cols_df.
  7. We concatenate with the main df, eliminate duplicates, and forward fill the newly concatenated columns.

    This gives us two continuous series with the following suffixes:

  8. '_chg': the price at the time the fractal was discovered
  9. '_fc': the fractal price at the time the fractal was discovered

Let's show what it looks like in practice:

t = 2df[-500:][['Close',f'abs Lo{t}', f'abs Hi{t}', f'abs HiLo{t}',f'abs HiLo{t}_fc', f'abs HiLo{t}_chg' ]].plot(figsize=(15,5), style= ['k','g.','r.','b:','m','c--'], title = f'{ticker} fractals {t}', grid=True)

This produces the following chart:

A graph showing a line graph AI-generated content may be incorrect.

Figure 4.5: Nissan fractals values at the time of discovery

This fractals method is not entirely foolproof. There are a few cases where the change seems off, especially when working with intraday short-term resolution where data quality sometimes leaves something to be desired. It does, however, do a quick and "easy" job.

The fractals_dates(df, col, col_name) is not useful when working with continuous data in production. We will look at a more efficient way to update the df later in this chapter.

Fractals do not confer regime. Fractals are merely swing highs and lows. As we will see in the next paragraph, combinations of those swings define regime. So, we replaced the single methodology score panel with volume. This produces the following chart:

A graph of a graph AI-generated content may be incorrect.

Figure 4.6: Nissan fractals highs and lows

Let's look at the code that produced the above chart:

subplots = [] ; pane = 1fractals_list = [2,3,4,5,6]mpf_relative(df[start_mpf:], rel=True, sma_list= sma_list, pane=1, color= 'k')for rel in rel_list:    _o,_h,_l,_c = rohlc(df,rel)       col_name = col_name_abs_rel(rel, name_abs = 'abs ', name_rel = 'rel ')      col = f'avg_{col_name}'    df[col] = avg_px(df, _h, _l, _c)    df = fractals_df(df, col, col_name)     print('fractals_df',df.shape)    df = fractals_dates(df,col,col_name)     mpf_fractals(df[start_mpf:],col_name, fractals_list, color_up, color_down, size, pane)         pane = 0mpf.plot(df[start_mpf:], type= 'ohlc', style= 'charles', figsize= (15,5), title =  f'{ticker}  {fields_list}', mav = tuple(sorted(sma_list)),             addplot= subplots, volume= True, volume_panel = 2, panel_ratios = (2,2,0.5,))

This is a little departure from the charts that precede and follow. We do not print the regime, so we need to reduce the subplots accordingly panel_ratios = (2,2,0.5,). Plotting all the fractals would create noisy charts. We select the levels we want to plot through the fractals_list.

The higher the level of abstraction, the more information fractals carry. We reflect this preeminence on the chart in two ways: size and opacity (alpha). Markers become bigger and darker as fractal levels increase.

Those fractals are very powerful. They constitute the basis for a lot of technical analysis. We can identify support, resistance, channels, trendlines, pennants, wedges, chart patterns, and pretty much the entire lexicon of technical analysis patterns using only level 1 or 2 fractals.

For now, we will focus on a classic pattern called higher highs, higher lows.

Higher highs, higher lows

This is another popular method. Stocks that trend up print higher highs and higher lows. A bullish market pulls both swing lows and swing highs upward. Conversely, stocks trending down make lower lows and lower highs in that order and therefore suggest sustained weakness. A bearish market pushes swing highs and lows downward. The theory makes intuitive sense. In practice, things are a little more complicated.

Markets sometimes print lower or higher lows or highs that throw calculations off before resuming their journey. Secondly, this method requires three conditions to be simultaneously met:

  1. A lower low
  2. A lower high
  3. Both lower low and lower high conditions must be met sequentially, which works only in orderly markets.

Those three conditions must be met consecutively in that precise order for the regime to turn bearish. Markets are more random and noisier than people generally assume. Let's explore a few scenarios and how we choose to deal with them:

  1. Markets do not always follow a neat succession of lower lows and lower highs on the bearish side, and vice versa on the bullish side. If the pattern is interrupted—for instance, with higher highs and lower lows—we have a choice: drop to neutral/inconclusive or do nothing. We choose to keep the regime as is until a pattern on the opposite side appears.
  2. In the case of an ongoing bearish trend, do we trail the market with the closest high, the one prior, or the initial highest high? Too tight, and the signal becomes too noisy. Too loose, and the signal loses its relevance.
  3. This is a continuation from the previous reset level. False positives happen when prices rise above the lower highs or drop below higher lows on the bearish and bullish sides, respectively. This can either be temporary or an early sign of an imminent regime change. The prudent thing to do is to change regime and revert if the price resumes its original trajectory. The regime definition gives the choice between either the latest low/high or the previous one. The default is two swings prior. This gives more stability to the regime definition.

There is no solution, only trade-offs. Tighter stops translate into higher transaction costs. Looser rules mean giving back a larger portion of profits. So, we decided to keep the code flexible and let readers choose. The variable shft is set at 2 by default for the penultimate ([n‑1]) swing.

In practice, we opted to give this regime definition a little more wiggle room to accommodate noise and randomness. Patterns can be disrupted and interrupted. The regime will not change until conditions on the other side are met. Without further ado, let's publish the code:

def higherhighs_df(df,_c,col_name,shft= 2):    v = 1    while (f'{col_name}Lo{v}' in df.columns) & (f'{col_name}Hi{v}' in df.columns):             lh = pd.DataFrame()        hl = f'{col_name}HiLo{v}_hl'        lhv = f'{col_name}LoHi{v}'        lh[lhv] = df[f'{col_name}Lo{v}'].sub(df[f'{col_name}Hi{v}'],fill_value =0).dropna()        lh.loc[(np.sign(lh[lhv]) < 0) &            (np.sign(lh[lhv] * lh[lhv].shift()) < 0) &            (-lh[lhv] < -lh[lhv].shift(2)) &            (lh[lhv].shift(1) < lh[lhv].shift(3)), hl] = -lh[lhv].shift(shft)        lh.loc[(np.sign(lh[lhv])>0) &            (np.sign(lh[lhv] * lh[lhv].shift()) < 0) &            (lh[lhv] > lh[lhv].shift(2)) &            (-lh[lhv].shift(1) > -lh[lhv].shift(3)), hl] = lh[lhv].shift(shft)        lh = lh.dropna(subset = [hl])             df[hl] = lh[hl]        df[hl] = df[hl].ffill()        df[ f'{col_name}HiLo_HH{v}' ] = np.sign(df[_c].sub(df[hl],fill_value=0))        v += 1    df = df.loc[:, ~df.columns.duplicated(keep='last')]    return df

This higherhighs_df() function uses the fractals to find higher highs, higher lows, lower lows, and lower highs. Let's break down the code:

  1. Loop through each fractal level using a while loop.
  2. Create a dataframe lh for lows and highs.
  3. distinguish lows from highs by assigning a negative value to the high series. Dropna.
  4. Bullish regime:
    1. Higher high
    2. Higher low, change on higher low
  5. Bearish regime:
    1. Lower low
    2. Lower high, change on lower high
  6. Assign low/high: note that we default to the previous low/high by shifting two values back, the penultimate swing.
  7. Slice lh df, drop one column.
  8. Join lh with the main df, propagate values using forward .ffill().
  9. Calculate the sign of the difference between the close price and the hilo values.
    1. When close is above hl, the regime is bullish
    2. When close is below hl, the regime is bearish

The main advantages of this method are entries and exits. On the long side, buy long on a low and exit on a high. On the short side, sell short on a high and exit on a low. These counter-trend entries and exits enable market participants to capture profits. Furthermore, stop losses are objectively defined at the higher low on the long side and at the lower high on the short side.

Next, let's plot the higherhighs_df() method:

  1. We will be using the mpf_fill. This function uses the score to assess whether the regime is in bullish or bearish territory. Every fractal contains some information. Rather than sticking with one level, such as 2 or 3, we create a weighted average df[f'{col_name}HiLo_HH'].
  2. We process the data through fractals_list [2, 3, 4, 5, 6]. The higher the level of abstraction, the lighter the shade.

This produces the following chart:

A graph of a graph AI-generated content may be incorrect.

Figure 4.7: Nissan higher highs

The preceding chart in absolute terms shows the lag occurring at higher levels of abstraction. While the regime had already turned bearish at levels 2 and 3, level 4 stayed bullish deep into a bearish phase. The simplest way to overcome the lag is to assign heavier weight to lower levels in the weighted average calculation.

Next, let's explore the cousin of the higher highs: the floor and ceiling. The following method uses the same swing highs and swing lows to define the regime in a much more powerful way. It is simple and statistically robust.

Floor and ceiling

The floor and ceiling method is originally a variation on the higher high/higher low methodology. Everyone has intuitively used it. And yet it is so obvious that no one has apparently bothered to formalize it. The floor and ceiling methodology is intuitive, yet computationally taxing.

Anyone who has spent some time on the markets has heard of the head and shoulders pattern. This looks like a peak (the head) surrounded by two lower highs (the shoulders). The floor and ceiling methodology focuses exclusively on the right shoulder. After all, there would not be a head if there were no left shoulder to begin with. Unlike the higher high/higher low method, only one of the two following conditions must be fulfilled for the regime to change:

  1. Bearish: A swing high must be lower than the peak.
  2. Bullish: A swing low must be higher than the bottom.

The classic definition is always valid regardless of the time frame and the asset class. There will be a trough followed by higher lows at the onset of every bull market. There will be a peak followed by lower highs to mark the beginning of a bear market. Those conditions are sufficient and necessary to qualify a regime change.

Randomness triggers exceptions that are handled in a simple, elegant way. There are two methods:

  1. Conservative: anchoring regime on floors and ceilings
    • If the regime is bearish and the price crosses over the ceiling, the regime turns bullish.
    • If the regime is bullish and the price crosses under the floor, the regime turns bearish.
  2. Aggressive: anchoring regime on the swing that enabled the regime change, referred to as the discovery swing.
    • If the regime is bearish and the price crosses over the discovery swing high, the regime turns bullish.
    • If the regime is bullish and the price crosses under the discovery swing low, the regime turns bearish.

This floor and ceiling method has only two regimes: bull or bear. A sideways regime is a pause within a broader bull or bear context. This method brings stability to regime definition. In practice, nothing is more costly than flip-flopping around a moving average. Stability enables market participants to better manage their positions.

The following function considers the desire for accuracy and the need for speed. In production, prices are refreshed continuously. The previous fractals_dates function becomes irrelevant. Fractals are recalculated every time a new bar appears. The regime will be the sign difference between the 'Close' price and the fractal value. If price penetrates the fractal, then this was not a valid swing high or low to begin with. The logic is sound and simple.

def floorceiling_df(df,_c,col_name):    '''    Regime is the sign difference between the close price and the base price.    It can be the floor and ceiling price or the average price at the time the fractal discovered later.    1. _fc (default when '_fc' is in the df.columns): Floor and ceiling price at the time the Floor and ceiling is discovered later (Built-in lag)    2. _chg: Average price at the time the fractal is discovered later (Built-in lag). Write 'chg' in the setting of the function to use this condition    3. else: Floor and ceiling price at the time of the floor and ceiling fractal. If none of the above conditions are met,            the default is the floor and ceiling price at the time of the fractal    '''    rg_FC_dict = {}    v= 2    while f'{col_name}HiLo{v}' in df.columns:                      if (f'{col_name}HiLo{v}_chg' in df.columns):            rg_FC_dict[f'{col_name}HiLo_FC{v}'] = np.sign(df[_c].rolling(2).mean().sub(                df[f'{col_name}HiLo{v}_chg'],fill_value = 0))             elif (f'{col_name}HiLo{v}_fc' in df.columns):            rg_FC_dict[f'{col_name}HiLo_FC{v}'] = np.sign(df[_c].rolling(2).mean().sub(                df[f'{col_name}HiLo{v}_fc'],fill_value = 0))                 else:            rg_FC_dict[f'{col_name}HiLo_FC{v}'] = np.sign(df[_c].rolling(2).mean().sub(                                                                    df[f'{col_name}HiLo{v}'], fill_value = 0))                         v+=1         df = pd.concat([df,pd.DataFrame(rg_FC_dict)], axis =1)    df = df.loc[:, ~df.columns.duplicated(keep='last')]    return df

This floorceiling_df() function has three levels of calculation. They are the following:

Columns with the suffix '_chg' show the price at the time the fractal was discovered. This lags the fractal by a fair amount of time and price discount. As a result, it is much closer to the current 'Close' price. Regime is therefore a lot more nervous. This configuration is an attempt at making the regime closer to the reality of trading the markets. Market participants act with imperfect information under stress. Note that the suffix '_chg' is given by the function fractals_dates(df, col, col_name).

If the suffix '_chg' is not present, the regime will be the sign difference between the fractal value and the current 'Close' price.

  1. Finally, when none of those suffixes are present, the regime will be the production default: the sign difference between price and fractal price. This is the default when using this function with continuous real-time prices in production.
  2. Calculate the 2-period rolling mean of close prices and subtract the appropriate baseline (average price or floor and ceiling level).
  3. Use np.sign() to convert differences to regime signals: +1 (bullish), -1 (bearish), 0 (neutral).
  4. Concatenate all regime columns to the DataFrame and remove duplicates.

Next, let's illustrate this with a chart. We recycle the same block of code to:

We recycle the same block of code to plot floor and ceiling regimes across fractal levels on a candlestick chart with volume and score panels.

A graph of a graph AI-generated content may be incorrect.

Figure 4.8: Nissan floor and ceiling

This regime definition is more nervous than the higher highs. The presence of a natural stop loss, e.g., the 'Close' price at the time the fractal was discovered, induces a lot of stops. In production, this can be circumvented by setting the default to the fractal price. This will also mean smaller position sizes and reduced risk appetite. There are no solutions, only trade-offs.

We have empirically calculated a composite score. It is time to formalize it into a proper function.

Composite score

"Paradoxically, the best way for a group to be smart is for each person in it to think and act as independently as possible."

– James Surowiecki, The Wisdom of Crowds

This is a substantial departure from the first version of this book. We really wanted to show the primacy of the Floor and Ceiling methodology. This signal universally works across a) asset classes and b) timeframes without c) additional need to reset the settings, and d) is more robust than classic methods. Furthermore, anyone from absolute beginner to market expert can intuitively understand this signal. At that time, it seemed like a good idea to proclaim the Floor and Ceiling as the winner of the regime definition contest.

A few years later, the thinking process has matured. The markets might be an epic adventure, but this is not a quest like The Lord of the Rings. There is NO "one signal to rule them all".

In reality, trading signals are more like French wines. French wines are famous for blending varietals to increase complexity and flavor. Chateauneuf-du-Pape blends as many as 13 varietals to come up with a unique, delicious flavor. Well, the more signals and methodologies we include in the mix, the more nuanced the regime definition might turn out. There is unquestionably value in blending signals. Perhaps we should go as far as to include countertrend or mean-reverting signals to account for euphoria or depression. Bottom line, a simple average of all methods is better than any single one in particular.

Our objective is to blend regime definition methods into one continuous score. The strength of the signal increases as more methods turn either bullish or bearish. Conversely, it weakens as the methods disagree.

We will start with the fractals. Regime is calculated at every fractal level. Each one carries some information. For example, one may choose to interpret each level on the daily chart as:

We will calculate a weighted average of all the methods. This is a 2-step process. First, we calculate the composite fractals regimes. Secondly, we calculate a composite across all regimes.

def dict_fractal(col_name, rg_method, fractal_dict):    '''    col_name: 'abs ' or 'rel '    rg_method: 'HiLo_FC' or 'HiLo_HH'    fractal_dict: for example {1: 0.1,2:0.4, 3:0.3, 4:0.2}    '''    return {f'{col_name}{rg_method}{v}':fractal_dict[v] for v in fractal_dict}

This function is a dictionary comprehension. We blend the column name, the regime method, and the fractal level with their corresponding weights.

  1. Accept column prefix (absolute or relative series), regime method (floor and ceiling or higher-highs), and fractal weight dictionary.
  2. Use dictionary comprehension to iterate through fractal levels and create full column names with their weights.
  3. Return mapping of regime column names to weights for downstream blending.

This produces a flexible weight mapping that enables weighted averaging of regime signals across multiple fractal levels and timeframes.

Next, let's build a function that calculates a weighted average of blended regime. This function can be used across multiple regime definition methodologies.

def rg_dict_blend(df, rg_dict):    blend = np.zeros(len(df))    n = 0    for rg in rg_dict:        if rg in df.columns:            if rg_dict[rg] > 0:                blend += df[rg] * rg_dict[rg]                n += rg_dict[rg]            else:                blend += df[rg]                n += 1    return round(blend / max(n,1),2)

This function calculates the weighted average of a regime dictionary. We iterate through the dictionary. If values are not null, then we append the weighted column to the composite weighted average.

  1. Initialize accumulator array and weight counter
  2. Iterate through regime columns in the dictionary, checking if each exists in the DataFrame.
  3. For positive weights, accumulate weighted regime values; for zero or negative weights, accumulate raw values with unit weight.
  4. Divide total by sum of weights (or 1 if no valid weights) and round to 1 decimal.

This produces a single blended regime score combining multiple signals with a flexible weighting scheme.

Next, let's calculate a dictionary score for all the regime definition methods:

for rel in rel_list:    print(rel, end = ', ')    _o,_h,_l,_c = rohlc(df,rel)    col_name = col_name_abs_rel(rel, name_abs = 'abs ', name_rel = 'rel ')    rg_cols_dict = {f'{col_name}BO{st}':0.1, f'{col_name}BO{lt}':0.2, f'{col_name}BO{xlt}':0.1,                f'{col_name}TT{st}/{lt}':0.2,                f'{col_name}SMA{st}/{lt}':0.2,                f'{col_name}EMA{st}/{lt}': 0.2,                f'{col_name}HiLo_HH':0.25,f'{col_name}HiLo_FC':0.25}    if rel == False:        score_abs_dict = rg_cols_dict    elif rel == True:        score_rel_dict = rg_cols_dictscore_dict = {**score_abs_dict, **score_rel_dict}df['score_abs'] = rg_dict_blend(df, score_abs_dict)df['score_rel'] = rg_dict_blend(df, score_rel_dict)df['score'] = rg_dict_blend(df, score_dict)df[['Close', 'score_abs', 'score_rel', 'score']].plot(figsize=(15,5), style = ['k', 'c-.', 'y:', 'm'], secondary_y = 'Close',grid=True, title = f'{ticker} score abs, rel and blended')

We take both functions for a spin. We will create a dictionary of regime methods and their corresponding weights. We then create a score for the absolute series and for the relative series. We then concatenate the two dictionaries, absolute and relative, into a composite score. We plot the absolute, relative, and overall scores along with the 'Close' price for the entire duration of the data frame.

We combine multiple technical indicators (breakouts, trend transitions, moving average crosses, higher-highs, floor, and ceiling) with predefined weights to generate absolute, relative, and composite regime scores.

  1. For each return series (absolute/relative), build a weight dictionary combining eight different indicators: breakouts (0.1-0.2 weights), trend transitions (0.2), moving average crosses (0.2 each), higher-highs (0.25), and floor and ceiling (0.25).
  2. Store absolute and relative weight dictionaries separately.
  3. Merge both into a composite dictionary.
  4. Apply rg_dict_blend() to compute three scores: absolute only, relative only, and combined.
  5. Plot all three scores against price to visualize regime alignment.

This produces three blended scores enabling multi-perspective regime confirmation: absolute for intrinsic moves, relative for benchmark-adjusted signals, and composite for a holistic view.

Let's visualize this with the following chart.

A graph showing a graph of data AI-generated content may be incorrect.

Figure 4.9: Nissan scores absolute, relative, and composite

This is "wisdom of the crowds" applied to the stock market. The blended score really captures market gyration. We assigned random weights and semi-random durations to the moving averages and breakout ranges. Still, the weighted average comes back with something half decent.

This composite score is an avenue worth exploring. Multiply this score by a position sizing algorithm and you will have a strength-adjusted position sizing in real time. Let's not get ahead of ourselves for now. We will revisit the idea further down the book. For now, let's apply what we have built so far to the entire population of stocks and store it in an arctic database.

Before we process the data for the entire tickers list, let's take a few minutes to set up a database that we will be reusing throughout the book.

ArcticDB set-up

Arctic database has been developed by the Man Group since 2012. It was released to the public in 2023. It is designed by quants for quants. arcticdb is open source (terms and conditions apply), fast, and easy to use. For more information on arcticdb, visit their GitHub page: . One major drawback is that arcticdb does not work with the latest Mac M series processors.

Without further ado, let's initialize an arcticdb database.

def initialise_adb_library_local(uri_path, library_name):    uri = f"lmdb://{uri_path}" # this will set up the storage using the local file system    ac = adb.Arctic(uri)    library = ac.get_library(library_name, create_if_missing=True,)# library_options=LibraryOptions(dynamic_schema=True))    return library

Defining arcticdb is simple and effortless, as we will see below.

Defining Arctic database utility functions for data retrieval

We will create helper functions to initialize Arctic database connections and retrieve single or multiple columns across multiple symbols efficiently.

  1. initialise_adb_library_local() connects to the Arctic database using local file system storage (LMDB) and creates the library if it is missing.
  2. adb_concat_single_column() filters symbols that exist in the library, reads a specific column for each symbol, renames columns by symbol name, and concatenates into a wide DataFrame.
  3. Functions enable efficient batch retrieval of historical data organized by symbol.

    We have now created reusable database connection and data extraction functions for working with Arctic's columnar storage across multiple assets.

  4. arcticdb stores data as {ticker:df}. If we want to retrieve one field, such as 'Close' or 'score', across the DB, we need to query ticker by ticker. So, let's build two simple functions that allow us to build dataframes. Let's start with a single field dataframe. 
    def adb_concat_single_column(library, symbols, column_name):    symbols_list = [symbol for symbol in symbols if symbol in library.list_symbols()]    comp_list_from_adb = [library.read(symbol, columns=[column_name]).data.rename(columns={column_name: symbol}) for symbol in symbols_list]    return pd.concat(comp_list_from_adb, axis=1)

This is predictably a list comprehension:

  1. We run a first list comprehension across the symbols to iterate on those present in the arcticdb.
  2. We read the column_name in a library for every symbol in a symbols list.
  3. We rename the data with the symbol.
  4. We concatenate the list into a dataframe.

Now, let's set up or initialize our arcticdb. Brace for impact:

library = initialise_adb_library_local('data', 'autos')

That's it. The database is stored in a folder called data. It sits on top of the current working directory. For more information on how to append, update, write, or slice arcticdb, please go to their GitHub page.

Now that we have built regime definition functions and set up a database, it is time to calculate regime across our investment universe.

Data manipulation across the investment universe

All the building blocks are in place. It is now time to run the regime definition functions across the entire investment universe.

We will use two ways to store information for the entire universe. One is a multi-index dataframe {ticker:df}. The other is arcticdb. We will also create a snapshot of every ticker's last row.

fractal_dict = {1: 0.1,2:0.4, 3:0.3, 4:0.2}df_dict = {} ; last_row_list= []for ticker in tickers_list:    raw_df = yf_droplevel(multiIndex_raw_data,ticker)    _o,_h,_l,_c = rohlc(raw_df,relative = False)    df = rel_fx(raw_df,_o,_h,_l,_c, bm_df, bm, ccy_df, tickers_fx_dict[ticker], start, end,rebase=True, mult=1)    print(ticker, df.shape, end =',')     for rel in rel_list:        _o,_h,_l,_c = rohlc(df,rel)        col_name = col_name_abs_rel(rel, name_abs = 'abs ', name_rel = 'rel ')        for slt in sorted(bo_list):            df[f'{col_name}BO{slt}'] = regime_breakout(df,_h,_l,slt)        df[f'{col_name}TT{st}/{lt}'] = turtle_trader(df, _h, _l, st, lt)         df[f'{col_name}SMA{st}/{lt}'] = regime_sma(df, _c, st, lt)        df[f'{col_name}EMA{st}/{lt}'] = regime_ema(df, _c, st, lt)                                                  col = f'avg_{col_name}'        df[col] = avg_px(df, _h, _l, _c)        df = fractals_df(df, col, col_name)        df = fractals_dates(df,col,col_name)             df = floorceiling_df(df,_c,col_name)        df[f'{col_name}HiLo_FC'] = rg_dict_blend(df,dict_fractal(col_name, 'HiLo_FC', fractal_dict))        df = higherhighs_df(df,_c,col_name,shft= 2)        df[f'{col_name}HiLo_HH'] = rg_dict_blend(df,dict_fractal(col_name, 'HiLo_HH', fractal_dict))                df['score_abs'] = rg_dict_blend(df, score_abs_dict)    df['score_rel'] = rg_dict_blend(df, score_rel_dict)    df['score'] = rg_dict_blend(df, score_dict)    df = round(df,2)    library.write(ticker, df)     last_row_list.append(last_row_cols_dict(df,ticker,df.columns))    df_dict.update({ticker: df.copy()}) multiIndex_df = multiindex_from_dict(df_dict, yf_format=False). round(2)

We are now processing the autos universe through a comprehensive regime analysis pipeline, generating breakout, trend, moving average, and fractal-based signals across absolute and relative returns, then storing results in the Arctic database.

Let's review the code step by step:

  1. Fractal_dict is the weighted average dictionary for the fractals. We define fractal weights (0.1, 0.4, 0.3, 0.2 for periods 1–4) and benchmark/currency settings.
  2. df_dict is the multiIndex dictionary. Last_row_list is the list of last_row dictionaries.
  3. We loop through the tickers list. We drop the level to retrieve OHLCV data and process the relative. For each ticker, we extract single-stock OHLCV and compute the currency-adjusted relative series normalized to the benchmark.
  4. We loop through the rel_list. We process the regime definition functions. For absolute and relative returns, we calculate 8 regime indicators (breakouts, turtle trader, SMA/EMA crosses, fractals, floor and ceiling, higher highs).
  5. We calculate the score based on the weighted average score_abs_dict and score_rel_dict we saw earlier. We write the df for each ticker in the arcticdb. For more information on how to append, update, or write arcticdb, please refer to the documentation.
  6. We append the last_row_list with a dictionary ticker, df.columns.
  7. We update the df_dict with a new dictionary {ticker,df}.
  8. Once the loop is finished, we create a multiindex_df from the df_dict. Note that the levels are inverse to the raw_df {ticker:df} generated from yfinance. To generate a multiindex df with {df:ticker}, switch the Boolean to True.

We have built two databases: one arcticdb and one multiindex dataframe. This investment universe is relatively small, so the information is still manageable. It will, however, rapidly become overwhelming.

Our objective as market participants is to triage information and concentrate our limited resources on the most promising candidates. So, let's print a quick snapshot of everyone's score:

last_row_df = pd.DataFrame(last_row_list).set_index('ticker')display_cols = ['date','score', 'score_rel', 'score_abs'] + list(score_dict.keys())score_sorted = last_row_df[display_cols].sort_values(by=['score'], ascending= True)

Extract the most recent regime scores across all tickers in the universe and rank them by composite signal strength to identify the strongest bullish and bearish opportunities. Let's break down this snippet of code:

  1. Convert the list of last-row metadata dictionaries into a DataFrame indexed by ticker.
  2. Select relevant columns: date, composite score, relative score, absolute score, and all individual indicator scores.
  3. Sort entire DataFrame by composite score in ascending order (bearish to bullish).

This gives a table of the detailed score at a specific date. At the time of writing, Renault and Volkswagen have scores of -0.9 and -0.82, respectively. They deserve a lot of attention.

Now, let's imagine the investment universe grows to 50 stocks. How could we find out the worst and top 3 scores?

pd.concat([score_sorted.head(3), score_sorted.tail(3)], axis=0)

This simple line of code would concatenate the top 3 and worst 3 into a single dataframe.

We would have the following order at the time of writing:

ticker

Score

score_rel

score_abs

RNO.PA

-0.9

-0.92

-0.88

VOW3.DE

-0.82

-0.88

-0.75

7201.T

-0.53

-0.58

-0.48

005380.KS

-0.07

0.41

-0.55

7203.T

-0.07

0.55

-0.68

F

0.72

0.62

0.82

Table 4.1: Score table

Voila! We have our work cut out for us. Our job is to understand why Mr. Market fails to appreciate European auto manufacturers Renault or Volkswagen for now.

Fundamental short selling starts where algorithmic short selling ends. The quantitative triage is done, and the detective work starts. Similarly, Ford seems to be outperforming. Our job is to understand why the market rewards Ford and decide whether it is worth going long.

Next, we will see how to work with fractals in production. Now that we have had a gentle practice run over a small population of stocks, let's expand the universe to the S&P 500.

Data manipulation across the S&P 500

In Chapter 3, we calculated a rudimentary bullish/bearish demarcation for the entire S&P 500 using a one-year breakout. Let's do it again with all the regime methodologies.

First, let's download the S&P 500 constituents from Wikipedia. If you want reliable data adjusted for survivorship bias, norgatedata.com is a reasonably priced and highly accurate data vendor.

url = 'https://en.wikipedia.org/wiki/List_of_S%26P_500_companies'headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64)}'}response = requests.get(url, headers=headers)df_SP500 = pd.read_html(response.text)[0]df_SP500 = df_SP500.rename(columns={'Symbol':'ticker', 'Security': 'name          ', 'GICS Sector':'sector','GICS Sub-Industry':'sub-industry'})df_SP500['ticker'] = df_SP500['ticker'].str.replace('.', '-', regex =False)df_SP500 = df_SP500.sort_values(by = ['sector','name']).set_index('ticker    ')

Extract a list of S&P 500 component stocks and prepare them for regime analysis alongside the benchmark. This is how we do it:

  1. We scrape the S&P 500 constituent list from Wikipedia with proper headers to avoid rejection. Note that it has inherent survivorship bias.
  2. We parse the HTML table and standardize column names (Symbol→ticker, Security→name, sectors).
  3. We normalize ticker symbols (replace '.' with '-' for compatibility with price data).
  4. We sort by sector and company name, then index by ticker for efficient lookups.
  5. We combine the benchmark ticker (^GSPC) with the constituent list for batch processing.

We have happily built a dataframe containing 500+ S&P 500 stocks organized by sector, ready for a multi-stock regime analysis pipeline.

Next, let's download historical OHLCV prices for the entire S&P 500:

tickers_list = list(df_SP500.index)[:]multiIndex_raw_data_SP500 = batch_df(tickers_list,batch_size, start, end,show_batch = False).round(2)multiIndex_raw_data_SP500.shape

Let's retrieve historical OHLCV data for all S&P 500 constituents plus the benchmark index for subsequent regime analysis.

  1. We construct a ticker list by combining the benchmark (^GSPC) with all 500+ S&P constituents.
  2. We call the batch_df() function to efficiently download price data across all tickers within the specified date range.
  3. We apply rounding to two decimal places for consistency and to reduce memory footprint.
  4. We verify the shape of the multi-indexed DataFrame to confirm successful retrieval across all securities.

We have now created a multi-indexed DataFrame containing OHLCV data for the S&P 500 universe, ready for regime indicator computation.

Let's initialize a new library called library_SP500:

library_SP500 = initialise_adb_library_local('data', 'SP500')

Let's run the regime loop across the entire population:

fractal_dict = {1: 0.1,2:0.4, 3:0.3, 4:0.2}bm_ticker = '^GSPC'   ;  bm = 'SP500' ; ccy = 'local' ; dgt = 2df_dict_SP500 = {} ; last_row_list_SP500 = []for ticker in tickers_list:    raw_df = yf_droplevel(multiIndex_raw_data_SP500,ticker)    _o,_h,_l,_c = rohlc(raw_df,relative = False)    df = rel_fx(raw_df,_o,_h,_l,_c, bm_df, bm, ccy_df, ccy, start, end,rebase=True, mult=1)     for rel in rel_list:        _o,_h,_l,_c = rohlc(df,rel)        col_name = col_name_abs_rel(rel, name_abs = 'abs ', name_rel = 'rel ')        for slt in sorted(bo_list):            df[f'{col_name}BO{slt}'] = regime_breakout(df,_h,_l,slt)        df[f'{col_name}TT{st}/{lt}'] = turtle_trader(df, _h, _l, st, lt)         df[f'{col_name}SMA{st}/{lt}'] = regime_sma(df, _c, st, lt)        df[f'{col_name}EMA{st}/{lt}'] = regime_ema(df, _c, st, lt)                                                  col = f'avg_{col_name}'        df[col] = avg_px(df, _h, _l, _c)        df = fractals_df(df, col, col_name)        df = fractals_dates(df,col,col_name)             df = floorceiling_df(df,_c,col_name)        df[f'{col_name}HiLo_FC'] = rg_dict_blend(df,dict_fractal(col_name, 'HiLo_FC', fractal_dict))        df = higherhighs_df(df,_c,col_name,shft= 2)        df[f'{col_name}HiLo_HH'] = rg_dict_blend(df,dict_fractal(col_name, 'HiLo_HH', fractal_dict))                df['score_abs'] = rg_dict_blend(df, score_abs_dict)    df['score_rel'] = rg_dict_blend(df, score_rel_dict)    df['score'] = rg_dict_blend(df, score_dict)    df = round(df,2)    library_SP500.write(ticker, df)     last_row_list_SP500.append(last_row_cols_dict(df,ticker,df.columns))        df_dict_SP500.update({ticker: df.copy()}) multiIndex_df_SP500 = multiindex_from_dict(df_dict_SP500, yf_format=False).round(2)

The loop takes a little less than 3 minutes on this time-honored, museum-worthy machine to calculate regime methods across 500+ securities over more than 10 years of daily data. It is not bad considering not only did we calculate fractals, but we also spent time looking for the time when they were discovered.

Let's take a look at the code.

Initialize fractal weight dictionary and storage containers for processed dataframes and metadata

  1. For each ticker, we extract single-stock OHLCV from multi-indexed price data and compute currency-adjusted returns normalized to the benchmark.
  2. - For absolute and relative return series, we calculate 8 regime indicators (breakouts, turtle trader, SMA/EMA crosses, fractals, floor and ceiling, higher-highs) across multiple timeframes.
  3. We apply weighted blending to create three composite scores: score_abs (absolute returns), score_rel (relative to benchmark), and score (combined).
  4. We write the complete processed DataFrame to the Arctic database for each ticker and store the latest metadata row for the universe scoreboard.
  5. We assemble all individual ticker DataFrames into a multi-indexed structure (ticker × columns) for universe-wide analysis and cross-sectional comparison.

We now have a multiindex DataFrame and an Arctic database complete with all signals for all constituents.

For the grand finale, let's calculate the sector rotation:

score_SP500_rel = adb_concat_single_column(library_SP500, tickers_list, 'score_rel')sector_avg_score_rel = bm_df[[bm]].join(sector_avg_df(df_SP500, 'sector', score_SP500_rel),how ='inner')sector_avg_score_rel.plot(figsize=(15,8),secondary_y=[bm], style = ['k'])    last_row_list_SP500.append(last_row_cols_dict(df,ticker,df.columns))        df_dict_SP500.update({ticker: df.copy()})

In , we touched on sector rotation. Now, we will use the mighty arcticdb to calculate sector rotation from all the scores in the index.

  1. We retrieve relative regime scores (score_rel) from the Arctic database for all S&P 500 tickers. We build a dataframe of relative scores across the arcticdb.
  2. We calculate the average relative score for all sectors and join with the benchmark. Note that we use the "inner" join pegged to the benchmark. It is, in fact, much easier to harmonize around the index than to let any component dictate the index. Besides, starting with the index makes it easier to color thereafter
  3. We plot the whole lot and, surprise, surprise, for the index "paint it black".

This produces a sector rotation chart showing which sectors are in regime phases (positive scores) or anti-regime phases (negative scores) relative to the S&P 500 benchmark.

Figure 4.10: S&P 500 & Sector rotation

Figure 4.10: S&P 500 & Sector rotation

That neurasthenic Rorschach chart is a clear upgrade from . We will revisit this arcticdb further along in the book. One thorny question remains. How do fractals fare in production? Fractals_df is a snapshot for the fractals at the last bar.

Next, let's explore how we can make use of fractals in production as fresh data keeps coming in, in real time.

Fractals in production

So far, we have been calculating fractals on historical datasets. fractals_df calculates fractals at a bar t. It has neither history nor memory. Fractals_dates is an attempt at finding when the fractals took place.

Production is a different story altogether. Data is refreshed continuously. We do not need fractals_dates when we can keep score from the previous bar to the current. All we need to do is compare the score before and after the data is refreshed. If it changes, then a new fractal was discovered.

Let's simulate a real-time price update and see how we can work with fractals in that environment.

We will query the Rolls Royce of short sellers from arcticdb. We will only extract OHLCV and average price in both absolute and relative terms. We will then set the clock back by slicing the dataframe to 550 or more periods ago.

ticker = '7201.T'_columns = list(df.columns[:11]) + ['avg_abs ', 'avg_rel ']data = library.read(ticker, columns=_columns, as_of=0).datafractal_dict = {1: 0.1,2:0.2, 3:0.3, 4:0.2}last_row_list = []n = len(data) + start_mpfwhile n <= len(data):    df = data.iloc[:n,].copy()    for rel in rel_list:        _o,_h,_l,_c = rohlc(df2,rel)           col_name = col_name_abs_rel(rel, name_abs = 'abs ', name_rel = 'rel ')        col = f'avg_{col_name}'                  df = fractals_df(df, col, col_name)        df = floorceiling_df(df,_c,col_name)        df[f'{col_name}HiLo_FC'] = rg_dict_blend(df,dict_fractal(col_name, 'HiLo_FC', fractal_dict))        df = higherhighs_df(df,_c,col_name,shft= 2)        df[f'{col_name}HiLo_HH'] = rg_dict_blend(df,dict_fractal(col_name, 'HiLo_HH', fractal_dict))         last_row_list.append(last_row_cols_dict(df,ticker,df_cols = list(df.columns)))    n += 1continuous_df = pd.DataFrame.from_dict(last_row_list).set_index('date').round(2)

Demonstrating real-time regime signal computation by incrementally processing historical data bar by bar, capturing how regime indicators evolve as new price data arrives.

  1. We read data from ArcticDB.
  2. We instantiate a last_row_list. We define fractal weights across timeframes (1-4 periods with specified weights).
  3. We set the clock 550 periods back at data +start_mpf.
  4. We loop while n is shorter than the length of the dataframe. We loop through data incrementally, starting from the earliest bar and extending the window by one bar per iteration.
  5. We create a copy of data of length n and process the fractals. For each window, compute all regime indicators (fractals, floor and ceiling, higher-highs) for both absolute and relative series.
  6. We take a snapshot of the last row of the df in a dictionary format. We extract the last row metadata (snapshot) at each iteration, including ticker, date, and all current signal values.
  7. We append the last_row_list.
  8. We create a continuous_df from the list of dictionaries.

This produces a complete time-indexed DataFrame showing how all regime indicators evolve continuously as new price data is processed, enabling analysis of signal timing and transitions.

When we collate all the individual snapshots into one consolidated DataFrame, fractals turn into lines. Individual dots are repeated snapshot after snapshot and become a continuous line. We can then plot fractals and score as seen in the following block of code:

continuous_df[['avg_abs ', 'abs Lo3','abs Hi3', 'abs HiLo_HH', 'abs HiLo_FC',]].plot(figsize=(15,4),            style = ['k', 'g-', 'r-','c:','y:', 'g','r',],            secondary_y = ['abs HiLo_HH', 'abs HiLo_FC'],            grid=True, title = f'{ticker}  continuous last row absolute level 3')

Displaying how absolute and relative regime indicators evolve over time, showing price levels, support/resistance bands, and higher-highs patterns with multi-panel visualization. Let's go through the code before visualizing the chart:

We plot the absolute return series with the price average (black), floor and ceiling bands (green/red), higher-highs pattern (cyan dotted), and floor/ceiling composite score (yellow dotted, secondary y-axis). Here is the corresponding chart:

A graph with different colored lines AI-generated content may be incorrect.

Figure 4.11: Nissan continuous last row absolute level 3

We plot average price, floor and ceiling, higher highs, and fractals highs/lows level 3.

Next, let's plot fractals highs and lows for the relative series level 2:

continuous_df[['avg_rel ', 'rel Lo2','rel Hi2', 'rel HiLo_HH', 'rel HiLo_FC',]].plot(figsize=(15,4),            style = ['k', 'g-', 'r-','c:','y:', 'g','r',],            secondary_y = ['rel HiLo_HH', 'rel HiLo_FC'],            grid=True, title = f'{ticker} continuous last row relative level 2')

We plot the relative return series at level 2 with similar indicators to compare relative performance against the benchmark.

A graph of a graph AI-generated content may be incorrect.

Figure 4.12: Nissan continuous last row relative, level 2

Fractals are stable considering the level of abstraction. We have generated three time-indexed charts showing continuous evolution of regime signals, enabling visual identification of regime transitions, support/resistance holds, and higher-highs patterns in real time. The objective was to demonstrate how to work with fractals with real-time prices.

  1. Run the fractals_df function.
  2. Calculate the composite score.
  3. Compare fresh data with the preceding one.

If there is a change, then one fractal was discovered. This method is particularly useful for finding higher levels of abstraction (3, 4, and beyond). Those higher fractals are discovered long after they occur. As such, they are quite hard to discover. The comparison using last_row_cols_dict(df,ticker,df_cols = list(df.columns)) simplifies the process and will recapture all fractals, regime changes, etc. In production, this is the fastest and most accurate method.

Next and final topic, let's look at fractals on intraday data.

Searching for the bar that triggered the bear avalanche

"A fractal is a way of seeing infinity."

– Benoit Mandelbrot

Every avalanche starts with a snowball. The search for the 5-minute bar that triggered the bear avalanche is the consecration of a twenty-year search. Indicators generally require parametrization. For example, the Relative Strength Index (RSI) default is 14 periods, MACD is (12,26), etc. Those parameters generally do not translate well from one timeframe to the next.

On the other hand, the fractals methodology does not require a default value. It preserves integrity across timeframes. It can be universally applied across timeframes and asset classes.

Fractals are like Matryoshka, Russian dolls. Our basic conjecture is that fractals at a fast timeframe can fold into a slower duration. One lower or higher price can trigger a fractal level 1, which in turn sets up a level 2, 3, 4, etc. So, could a fractal 1 on a 5-minute bar leading to a level 5, 6, 7 be the equivalent of a level 3 daily bar? If that is the case, we could potentially have found the bar that triggered the bear avalanche. Let's put that theory to the test. We will need as much historical low-resolution data as possible. After all, we want to know how many levels of abstraction a 1-minute bar can pack into a 20-year-long span. Then, we want to resample the data to higher resolution, like 5-minute, all the way up to daily and weekly bars. If the low resolution on high levels looks like the high resolution on lower to middle levels, then we are clearly on to something. This means there is integrity along the time continuum. Enough with the theory, let's get down to the code.

One-minute data resampling

We need some long-term data to calculate fractals. The ultimate weapon of choice for long-term accurate data is this website .

Eodhd has 30 years of history with 1-minute bars for the entire S&P500. Data is clean and generally accurate. Eodhd is to retail traders what Bloomberg is to institutions. Besides, if you want to test strategies where you will put your hard-earned savings, you need accurate data. is well worth the investment both for testing and production for anyone who is serious about trading.

For the purpose of this book, we will use freely available data so that anyone can replicate the code without incurring undesired costs. Fortunately, there is a website that provides 1-minute data for free for limited dataset. Shout out to the eponymous website: .

Below is the resampling function we will be using:

def resample_ohlcv(df, rel, period ):    _o,_h,_l,_c = rohlc(df,relative = rel)    ohlcv_dict = {_o: 'first',_h: 'max',_l:'min',_c: 'last','volume': 'sum'}    intraday_dict = {dct : 'mean' for dct in df.columns[5:]}    ohlcv_dict.update(intraday_dict)    df_resampled = df.resample(period, closed='left', label='right').apply(ohlcv_dict)    df_resampled = df_resampled.dropna(subset = [_o,_h,_l,_c])    return df_resampled

Creating a reusable function to resample one-minute OHLCV data to higher timeframes.

  1. Let's go through the code: We create a dictionary for ohlcv. We define aggregation rules: Open (first), High (max), Low (min), Close (last), and Volume (sum)
  2. We update with the average of all subsequent columns. We extend the aggregation dictionary to include all intraday indicator columns with mean aggregation.
  3. We resample with the corresponding period and apply the ohlcv_dict.
  4. We dropna along the ohlc column to ensure clean, upsampled data.

We have created a flexible resampling function that enables analysis across multiple timeframes (1-min to weekly) with consistent OHLCV aggregation and indicator preservation.

Next, we build a gigantic dataframe of 1-minute data directly from the internet.

one_minute_weblinks = [ # Website source data: 'https://oneminutedata.com/spy-intraday-data''https://indigo-gazelle447398.hostingersite.com/data/spy/spy_2024_6e0851922b721890a49c6a240c230ba39670d1a7.csv.gz', 'https://indigo-gazelle-447398.hostingersite.com/data/spy/spy_2023_703d78ef9b61798e6de7dd6bf8d8a668d51e5aaa.csv.gz', 'https://indigo-gazelle-447398.hostingersite.com/data/spy/spy_2022_b0cb458763c66c311a46aa1f8b8549ded4e9da6f.csv.gz', 'https://indigo-gazelle-447398.hostingersite.com/data/spy/spy_2021_b65d518b0047dd8eaa2c2c3a98c6f61ea8637569.csv.gz', 'https://indigo-gazelle-447398.hostingersite.com/data/spy/spy_2020_cdb851e720986036f80db3aeeb5943e11490f10f.csv.gz', 'https://indigo-gazelle-447398.hostingersite.com/data/spy/spy_2019_82b12186b367ab8006e0bf6c51c61f61a471e7ff.csv.gz', 'https://indigo-gazelle-447398.hostingersite.com/data/spy/spy_2018_8347579bc522cc3617ace45ff3f81dc0df5a04d7.csv.gz', 'https://indigo-gazelle-447398.hostingersite.com/data/spy/spy_2017_28c365f5e53b09b163ca8f36d62d82ae434ca538.csv.gz', 'https://indigo-gazelle-447398.hostingersite.com/data/spy/spy_2016_c39e8ddcd9bced3a283b987d1ef91048ccfbe6bd.csv.gz', 'https://indigo-gazelle-447398.hostingersite.com/data/spy/spy_2015_736c7bd2f88a7bdcd476fbdf0dedfd66b8c217a2.csv.gz', 'https://indigo-gazelle-447398.hostingersite.com/data/spy/spy_2014_a8e5ee2292b7ebd074baf6720412476ecc19c441.csv.gz', 'https://indigo-gazelle-447398.hostingersite.com/data/spy/spy_2013_e8bb4ae219ec14fcbe73ff011b6c537ba0b25fd7.csv.gz', 'https://indigo-gazelle-447398.hostingersite.com/data/spy/spy_2012_a89a832898914f255e370b16908f536ff851538f.csv.gz', 'https://indigo-gazelle-447398.hostingersite.com/data/spy/spy_2011_e63fe48d566afde3820631656e6cf2b313a71e00.csv.gz', 'https://indigo-gazelle-447398.hostingersite.com/data/spy/spy_2010_16cd469f7011689f465dc50e13a6076853745de6.csv.gz', 'https://indigo-gazelle-447398.hostingersite.com/data/spy/spy_2009_3aa7d5c5434c69cb4ad89073c9cfe5a22db2713f.csv.gz',]                    historical_df_1min = pd.DataFrame()for weblink in one_minute_weblinks:    df = pd.read_csv(weblink, index_col=0, parse_dates= True,compression = 'gzip')    historical_df_1min = pd.concat([historical_df_1min, df], axis =0)    print(df['close'][-1:])historical_df_1min.shape

This looks impressive, but the code is quite rudimentary:

  1. We create a list of weblinks for all the files. We define a list of web links to gzip-compressed CSV files containing SPY one-minute data spanning 2009-2024 from .
  2. We loop through the list and read_csv with a compression format gzip, parse dates, and parse the first column as DatetimeIndex.

We concatenate each year's DataFrame vertically (axis=0) into a single time-indexed dfNext. Now, let's create a dictionary of resampled dataframes and process fractals for each period:

rel = False ; _o,_h,_l,_c = rohlc(df,rel)  ;    col_name= '' ;  col = f'avg'rg_dict = {t : round(t*0.1,1) for t  in range(1,16)}df_dict = {}interval_list= ['1min','2min','3min','4min','5min','10min','15min','20min','30min','1h','4h','1d','1W']df_1min = historical_df_1min.copy()for interval in interval_list:    if interval != '1min':             df_dict[interval]= resample_ohlcv(df_1min, rel= False, period = interval)    else :        df_dict[interval]= df_1min.copy()    print(interval, df_dict[interval].shape, end = " ")     df_dict[interval][col] = avg_px(df_dict[interval], _h, _l, _c)    df_dict[interval] = fractals_df(df_dict[interval], col, col_name)    df_dict[interval] = fractals_dates(df_dict[interval],col,col_name)         df_dict[interval] = floorceiling_df(df_dict[interval],_c,col_name)    df_dict[interval][f'{col_name}HiLo_FC'] = rg_dict_blend(df_dict[interval],dict_fractal(col_name, 'HiLo_FC', fractal_dict))    df_dict[interval] = higherhighs_df(df_dict[interval],_c,col_name,shft= 2)    df_dict[interval][f'{col_name}HiLo_HH'] = rg_dict_blend(df_dict[interval],dict_fractal(col_name, 'HiLo_HH', fractal_dict))    print(df_dict[interval].shape)

Building a multi-timeframe regime analysis by resampling one-minute SPY data across 13 intervals (1-min to weekly) and computing fractals, floor and ceiling, and higher-highs patterns for each interval.

  1. We instantiate the usual variables _o, _h, _l, _c and , rel.
  2. We create a fractal dictionary and increase the weight by 0.1 for every level. The objective is not to get a precise weight, just to assign values.
  3. We instantiate a df_dict that will contain all the dataframes.
  4. We create an interval list spanning 1-minute to weekly timeframes: '1min', '2min', '3min'... '1W'.
  5. We loop through the interval_list. For each resampled timeframe, we compute average price, fractals at multiple levels, floor and ceiling bands, and higher-highs patterns.
  6. We process fractals and regime and populate the df_dict. We blend fractals across levels using a predefined weight dictionary for composite scores at each timeframe.

We have built a dictionary containing 13 resampled DataFrames with complete regime indicators across all timeframes, enabling multi-scale analysis from intraday to weekly.

Next, let's plot the 5-minute along with the daily charts:

interval = '5min'df_dict[interval][['HiLo3',                   'HiLo4','HiLo5','HiLo6', 'HiLo7',]].plot(    figsize=(20,3), secondary_y = ['close'], grid=True,style = ['y','c','g','m','b'], title = interval)interval = '1d'df_dict[interval][['HiLo1','HiLo2','HiLo3','HiLo4']].plot(    figsize=(20,3), secondary_y = ['close'], grid=True,style = ['c','g','m','b'], title = interval)

Visualizing fractal-based regime indicators across different lookback periods for both intraday (5-minute) and daily timeframes to identify signal behavior.

  1. We select the 5-minute interval and plot fractal levels 3-7 (HiLo3 through HiLo7) to visualize multi-period regime signals.
  2. We apply distinct color styling (yellow, cyan, green, magenta, blue) for visual differentiation.
  3. We select the daily (1d) interval and plot fractal levels 1-4 (HiLo1 through HiLo4) showing longer-term regime patterns.
  4. We display both charts with labeled titles to enable direct comparison of signal behavior across timeframes.

We generated a side-by-side visualization of multi-level fractal regime signals for intraday and daily data, revealing how different lookback periods capture regime transitions at various timescales.

This produces two charts:

A line of a graph AI-generated content may be incorrect.

Figure 4.13: 5-minute chart with fractals levels 3 to 7

Now, let's compare the above graph with the daily chart.

A graph with colored lines AI-generated content may be incorrect.

Figure 4.14: 1-day chart with fractal levels 1 to 3

Those charts look eerily similar. The higher the level of abstraction at the 5-minute bar, the more similar it looks to the daily chart. It seems this 20-year mission may finally come to a happy ending. Next, let's zoom!

chart_start = -35000chart_end = -5000interval =  '5min't = 4df_dict[interval][chart_start:chart_end][['close','Hi2', 'Lo2','Hi3', 'Lo3','Hi4', 'Lo4','Hi5', 'Lo5', f'HiLo{t}',f'HiLo{t}_fc',f'HiLo{t}_chg',                           f'HiLo{t+1}',f'HiLo{t+1}_fc',f'HiLo{t+1}_chg',                        f'HiLo{t+2}',f'HiLo{t+2}_fc',f'HiLo{t+2}_chg',]].plot(figsize=(18,6), secondary_y = [f'HiLo_FC{t}'], grid=True, style = ['grey',                                           'r.','g.','ro','go','gv','r^','bv','m^', 'y:','y','y-.',:','c','c-.','b:','b','b-.',], title = f'{interval}: HiLo{t} HiLo{t}_fc HiLo{t}_chg, HiLo{t+1} HiLo{t+1}_fc HiLo{t+1}_chg, HiLo{t+2} HiLo{t+2}_fc HiLo{t+2}_chg')

Creating a zoom view that shows how consecutive fractal levels capture regime changes at 5-minute resolution over a specific chart window.

  1. We define a chart window as the last 35,000 to 5,000 bars to focus on recent significant price action.
  2. We select the 5-minute interval starting from fractal levels 4, 5, and 6 for multi-scale comparison.
  3. We plot close price (grey), support/resistance levels (red/green dots and symbols), and fractal composites (yellow, cyan, and blue with different line styles).
  4. We include three signal types per fractal level: HiLo composite (dotted), floor and ceiling band (solid), and change indicator (dash-dot).
  5. We use distinct color and symbol combinations (dots, circles, triangles) to distinguish Hi/Lo bands and level progression.
  6. Secondary y-axis for floor and ceiling composite to prevent scale clipping of price action.

This produces a comprehensive 5-minute chart revealing fractal-based support and resistance zones and their transitions, showing multi-level regime signal coordination.

We cropped the duration to make it more visible. We will publish the 1d code and then print the charts back-to-back.

chart_start = -600chart_end = -60interval =  '1d'   ;t = 2df_dict[interval][chart_start:chart_end][['close','Hi2', 'Lo2','Hi3', 'Lo3','Hi4', 'Lo4', f'HiLo{t}',f'HiLo{t}_fc',f'HiLo{t}_chg',                         f'HiLo{t+1}',f'HiLo{t+1}_fc',f'HiLo{t+1}_chg',                       f'HiLo{t+2}',f'HiLo{t+2}_fc',f'HiLo{t+2}_chg',]].plot(figsize=(18,6), secondary_y = [f'HiLo_FC{t}'], grid=True,style = ['grey', 'r.','g.','ro','go','gv','r^', 'y:','y','y-.', 'c:','c','c-.', 'b:','b','b-.',], title = f'{interval}: HiLo{t} HiLo{t}_fc HiLo{t}_chg, HiLo{t+1} HiLo{t+1}_fc HiLo{t+1}_chg, HiLo{t+2} HiLo{t+2}_fc HiLo{t+2}_chg')

Let's start with the 5-minute bar:

A graph with red and green lines AI-generated content may be incorrect.

Figure 4.15: 5-minute chart with fractals levels 4 to 6

Next, let's compare with the daily chart.

A graph with numbers and lines AI-generated content may be incorrect.

Figure 4.16: 1-day chart with fractals levels 2 to 4

The fractals on the 5-minute chart look pretty much like the ones on the daily bar two levels lower. For example, the 5-minute level 5 and 6 look like the level 3 and 4 on the daily chart. They were discovered on the same day at the same price.

At this juncture, we could put this quest to rest. We could find the exact 5-minute bar at level 1 that triggered the level 6 change at the daily level 3. And we did. Mission accomplished.

This sends multi-timeframe trading into a whole new multiverse of possibilities. 5-minute charts fold neatly into 15, 30, 1 hour, 4 hours, and daily charts. The peace of finding the 5-minute bar that triggered the bear market lasted less time than an inefficiency on the S&P 500 futures. It promptly prompted the next question. If we trade the exact same strategy with the exact same parameters on the same instrument, then what asset class are we exactly trading? Drum roll: TIME. We could take the red pill and see how deep the rabbit hole goes, but it would lead us far beyond the scope of this book.

Summary

We have covered a lot of ground in this chapter. We have examined a few regime methodologies that will help you pick up on signals that a market is going up or down. Regime breakouts and moving average crossovers are staples in the arsenal of trend-following traders. Duration is as much a function of style as of what the market happens to reward. Then, we introduced the higher highs and floor and ceiling methodologies. Those regime definitions work on absolute and relative series. They are symmetrical and tend to be more stable than other methodologies.

Regime definition methodologies are not mutually exclusive. For example, the floor and ceiling method could be used to determine the direction of trades, long or short. Then, regime breakout could be used to enter after consolidation or sideways markets. Finally, the moving average crossover could be used to exit positions. All these signals could be blended into a continuous weighted average score. The average of all opinions is statistically more robust than any one method in particular.

In this chapter, we defined a methodology that remains congruent from short time intervals to daily bars and beyond without the need for additional settings. We invite you to explore the possibilities generated from the fractals methodology. There is so much work to be done, so much potential for heartbreaks, disappointments, frustration, and ultimately breakthroughs. Do not hesitate to reach out to me.

Having a signal is one thing. Turning it into a profitable strategy with a robust statistical edge is another. There is no profitable equivalent to the "buy-and-hold" mentality on the short side. Short selling is like mixed martial arts. That belt will claim its pound of flesh. For this reason, the next chapter will focus on developing a robust trading edge.

Get this book's PDF copy, code bundle, and more

Scan the QR code (or go to ). Search for this book by name, confirm the edition, and then follow the steps on the page.

Image

Image

Note: Have your invoice handy. Purchases made directly from the Packt website don't require an invoice.

Назад: Part 2: The Outer Game: The Trading Edge
Дальше: Chapter 5: The Trading Edge Is a Number, and Here Is the Formula