A Guide to Model Selection Part VI: Frequency, latency, and right-sized models

In this part, we will talk about the two most critical factors you must consider before putting your model to work: Frequency and Latency.

What is Frequency? (The Input Speed)

Frequency describes how often your model receives data. It is the “Heartbeat” of your information stream.

• High Frequency: In high-speed trading or sensor monitoring, your model might receive an input every 2 ms.
• Low Frequency: In business reporting or climate tracking, the model might only get an input once a month.

What is Latency? (The Reaction Speed)

Latency is the “Delay.” It is the time it takes from the moment the model receives an input to the moment it delivers an output.

• The Expectation: We need to know if the model can process the data within the time frame we expect.
• The Stake: If a self-driving car has a latency of 2 seconds to recognize a “Stop” sign, it is already too late.

The “Right-Sized” Model

If we have a super-smart model that knows everything, why don’t we use it every time?

The answer is Balance. Using a massive, heavy model for a fast task is like using a semi-truck to deliver a single pizza. It is powerful, but it is too slow to move, and it costs too much to run.

The Conflict: Speed vs. Accuracy

In the real world, you usually have to choose: Do you want it to be perfect, or do you want it right now?

• The Heavy Model: This is 99% accurate, but it takes 100ms to think. If your data comes in every 10ms (High Frequency), this model will fail. It can’t keep up, and the data will pile up until the system crashes.
• The Right-Sized Model: This might be 92% accurate, but it only takes 5ms to think. It is fast enough to finish the job before the next input arrives.

Where does it live? (The Container)

You also have to consider where the model is running.

• On the Cloud: You have big power, so you can use a “Heavy” model.
• On the Edge (Mobile/Camera): You have a small “container.” Your model must be “Right-Sized” to fit the memory and battery of that device.

The Rule for Selection:

A model is only “Smart” if it can give you the answer within the time frame you expect. If the answer comes too late, it doesn’t matter how accurate it was.

When we apply these rules to the previous chapters, we see that there are many factors to consider. Choosing the right AI isn’t just about the “best” model; it is about the “right” model for your specific situation.

To choose the right model, you don’t just pick the newest AI. You follow a logical path based on the factors we’ve discussed. Use this checklist to build your blueprint:

1. Intention Choice: Is the goal Forecasting (predicting the future) or Classification (identifying a label)?
2. Memory Type: Based on that intent, do you need Stateful thinking (needs the “story”) or Stateless thinking (only needs the “now”)?
3. Speed Requirements: Look at your Frequency (how often data arrives) and your Latency (how fast you need an answer). This tells you if you need a “Lightweight” model or can afford a “Heavy” one.
4. Look-back Window: How much history matters? Do you need a Long Window to see big trends or a Short Window for quick reactions?

graph TD
    Start[AI Model Selection Checklist] --> Q1{1. Intention}
    
    Q1 -->|Future Trend| F[Forecasting]
    Q1 -->|Identify Label| C[Classification]
    
    F --> Q2
    C --> Q2
    
    Q2{2. Memory Type}
    Q2 -->|Stateful| M[Needs 'The Story' / History]
    Q2 -->|Stateless| NM[Needs 'The Now' / Static]
    
    M --> Q3
    NM --> Q3
    
    Q3{3. Speed & Frequency}
    Q3 -->|High Latency / Low Freq| Heavy[Heavyweight / High Precision]
    Q3 -->|Low Latency / High Freq| Light[Lightweight / Real-time]
    
    Heavy --> Q4
    Light --> Q4
    
    Q4{4. Look-back Window}
    Q4 -->|Long Window| LW[Long-term Trends / Transformers]
    Q4 -->|Short Window| SW[Immediate Reactions / CNN-1D]
    
    LW --> Finish[Final Blueprint Built]
    SW --> Finish

    %% Styling for "Strong" finish
    style Start fill:#1a1a1a,stroke:#facc15,stroke-width:2px,color:#fff
    style Finish fill:#b45309,stroke:#fff,stroke-width:2px,color:#fff

Putting it into Practice (The "Fall Detection" Example)

Imagine you are building an AI for a security camera to detect if an elderly person falls. Here is how your checklist would look:

1. Intention: Classification (You want to label the event as “a fall”).
2. Memory: Stateful (You need to see the “movement” over time to know it’s a fall).
3. Frequency/Latency: High Frequency & Low Latency (You need to process many frames per second and alert someone immediately).
4. Window: Short Window (You only need the last 2–3 seconds of video to see the fall).

The Result: You would choose a Lightweight, Stateful model with a Short Look-back Window.

We have covered a lot of ground in this series. We moved from understanding Time Series data to choosing a Brain (Memory), defining our Intent, and finally balancing Speed (Frequency/Latency).

In the next part, we will explain why some models fail silently when working with time-series data.