Building Server Driven Smart Filter With Syntax Highlighting and Intellisense
If you are reading this then most probably you are a developer and most probably you have experience with IDEs. IDEs have become indispensible for developers, where smart intellisense suggestions simplify your life. I wanted that same magic for my web app's search functionality. Here's how I built an intelligent search system using CodeMirror on the frontend and F# doing the heavy lifting on the backend.
The Problem
I wanted users to filter cryptocurrencies using natural language like name:Bitcoin marketCap>1B OR ticker:ETH. I wanted search to be smart enough to suggest completions, highlight syntax errors, and handle complex queries without breaking.
After few iterations I arrived at the reasonable model. Keep minimal logic with syntax highlighting based on codemirror on frontend, and use F# where it shines, for both intellisense and expression translation into AST.
Codemirror is my favourite go-to editor, which is heavily customizable and browser-based. You could compare it to vscode, which is very heavy and you would have a hard time customizing it into a single line text-like field.
The Frontend: Making It Look Smart
Grammar Definition
CodeMirror needs to know what your language looks like. Here's my grammar for filter queries:
@top Query {
Clause (OrKeyword Clause)*
}
Clause {
StringClause | NumericClause
}
StringClause {
Keyword Colon Symbol
}
NumericClause {
Keyword ComparisonOperator NumberLiteral
}
Symbol { (AlphaNum | SpaceChar)+ }
@tokens {
Keyword { "name" | "ticker" | "marketCap" | "bidSize" | "bidPrice" | "askSize" | "askPrice" | "volume24h" | "spread" | "midPrice" | "return1m" | "return5m" | "return60m" | "returnDay" | "returnWeek" | "returnMonth" | "sharpeRatio" | "calmarRatio" | "annualizedReturn" | "volatility" | "maxDrawdown" | "currentDrawdown" | "active" | "since" | "until" }
OrKeyword { "OR" | "or" }
Colon { ":" }
ComparisonOperator { ">" | "<" | "=" }
AlphaNum { $[A-Za-z0-9*().]+ }
SpaceChar { "_" | "-" }
NumberLiteral { $[0-9]+ ("M" | "B" | "T" | "K" | "m" | "b" | "t" | "k")? }
space { @whitespace+ }
@precedence {
Keyword, OrKeyword,
AlphaNum, SpaceChar, NumberLiteral
}
}
@skip { space }
Simple but powerful: field filtering (name:Bitcoin), logical operations, and human-readable numbers (1B instead of 1000000000).
The Completion System
Here is the simple create completions closure that creates an observable. It is nicely throttled (this not overwhelming the backend) and delegates heavy lifting to the server.
export function createApiCompletions(
client: HttpClient,
destroyRef: DestroyRef,
) {
const requestSubject = new Subject<CompletionContext>();
const throttledStream = requestSubject.pipe(
takeUntilDestroyed(destroyRef),
debounceTime(100),
switchMap((context) => {
const pos = context.pos;
const doc = context.state.doc.toString();
return intellisense(client, doc, pos).pipe(
catchError((err) => {
console.log(err, "error");
return of(null);
}),
);
}),
share(),
);
return async function completions(
context: CompletionContext,
): Promise<CompletionResult | null> {
const resultPromise = firstValueFrom(throttledStream);
requestSubject.next(context);
const result = await resultPromise;
return result ? replaceWhitespace(result) : null;
};
}
Why this works: debounced API calls, graceful error handling, and the backend gets full context about what the user is typing.
The Backend: Where the Magic Happens
Why Not Client-Side?
Simple: the backend knows things the frontend doesn't. Real instrument names from the database, complex business rules, and robust parsing with FParsec. Plus, malicious queries get caught server-side where they belong.
Two Tools, Two Jobs
I decided to split backend into 2 parts. Parser combinators excel at translating complete expression into AST. However they have a hard time with incomplete, unfinished input.
Regex-based active patterns handle messy, incomplete input while you're typing. FParsec validates complete queries with proper error messages. Right tool for the right job.
Active Patterns: The Secret Sauce
F#'s active patterns make parsing incomplete user input surprisingly elegant:
let suggest (suggestions: Suggestions) (input: string) (pos: int) =
let options: Completion list =
match beforeCursor with
| Utils.RegexStringFieldAny suggestions.ChoiceFields (field, items, matched) ->
items
|> List.filter (fun x -> fuzzyMatch x matched)
|> List.map (fun x ->
{ Label = x
Apply = $"{field}:{x} "
Type = "constant"
Boost = None
Detail = None })
| Utils.RegexNumericAny suggestions.ComparisonFields (field, matched) ->
match matched with
| None ->
[ ">"; "<"; "=" ]
|> List.map (fun op ->
{ Label = op
Apply = $"{field.Name}{op}"
Type = "operator"
Boost = None
Detail = None })
| Some(op, number) ->
field.Values
|> List.filter (fun x -> fuzzyMatch x number)
|> List.map (fun num ->
{ Label = num
Apply = $"{field.Name}{op}{num} "
Type = "number"
Boost = None
Detail = None })
| _ ->
let keywordOptions =
suggestions.Keywords |> List.filter (fun kw -> fuzzyMatch kw.Label currentWord)
keywordOptions
These patterns automatically detect what the user is trying to type:
let (|RegexStringFieldAny|_|) (fieldConfigs: Field list) (input: string) =
let tryField (field: Field) =
match (|RegexMatch|_|) $"{field.Name}:([^\s]*)$" input with
| Some m -> Some(field.Name, field.Values, m.Groups[1].Value)
| None -> None
fieldConfigs |> List.tryPick tryField
let (|RegexNumericAny|_|) (fieldConfigs: Field list) input =
let tryField (field: Field) =
let fieldWithOpPattern = $"{field.Name}([=<>])([^\s]*)$"
match (|RegexMatch|_|) $"{field.Name}$" input with
| Some _ -> Some(field, None)
| None ->
match (|RegexMatch|_|) fieldWithOpPattern input with
| Some m -> Some(field, Some(m.Groups[1].Value, m.Groups[2].Value))
| None -> None
fieldConfigs |> List.tryPick tryField
- String fields:
name:Bitc→ suggests Bitcoin, BitcoinCash, etc. - Numeric fields:
marketCap>→ suggests operators and values - Space separation: Natural whitespace-separated expressions
FParsec: Making Complex Parsing Look Easy
Parser combinators sound scary but they're just Lego blocks for parsing:
The Building Blocks
type Clause =
| NameClause of string
| TickerClause of string
| ActiveClause of bool
| MarketCapClause of ComparisonOperator * float
| Return1m of ComparisonOperator * float
// ... more clauses
type Query =
| AndQuery of Query * Query
| OrQuery of Query * Query
| SingleClause of Clause
| EmptyQuery
Readable Parsing
Look how clean this gets:
let numberWithSuffix: Parser<float, unit> =
pipe2 pfloat (opt (anyOf "KkMmBbTt")) (fun value suffix ->
let multiplier =
match suffix with
| Some 'K' | Some 'k' -> 1e3
| Some 'M' | Some 'm' -> 1e6
| Some 'B' | Some 'b' -> 1e9
| Some 'T' | Some 't' -> 1e12
| None -> 1.0
| _ -> 1.0
value * multiplier)
let numericFactory field clause =
keyword field >>. comparison .>>. numberWithSuffix |>> clause
let market = numericFactory "marketCap" MarketCapClause
let return1m = numericFactory "return1m" Return1m
Handling Complex Queries
Boolean logic that actually makes sense:
let query clause =
let pSingleAsQuery = clause |>> SingleClause
let orOpToken = keyword "OR" <|> keyword "or"
let orOperator =
attempt (ws1 >>. orOpToken .>> ws1) >>% fun left right -> OrQuery(left, right)
let orExpr = chainl1 pSingleAsQuery orOperator
let andOperator =
attempt (ws1 >>. notFollowedBy orOpToken)
>>% fun left right -> AndQuery(left, right)
chainl1 orExpr andOperator
Result: queries like name:Bitcoin marketCap>1B OR ticker:ETH return1m>0.05 just work.
The Bottom Line
You don't need AI or complex client-side libraries to build smart search. Just split the work sensibly: pretty editing in the browser, smart logic on the server.
The magic formula:
- Backend knows your data and business rules
- Active patterns handle incomplete input gracefully
- FParsec validates complete queries properly
- Users get natural, readable query syntax
- You as a developer keep your sanity
Want to see this in action? Check out the live demo at cryptoquant.dev.