module FuzzyMatch
Overview
Fuzzy string matching for Crystal.
Every scorer compares two strings and returns a similarity between 0.0
(nothing in common) and 100.0 (identical). The scorers follow the
definitions used by Python's fuzzywuzzy and rapidfuzz, so scores carry
over if you already have thresholds tuned for those libraries.
FuzzyMatch.ratio("this is a test", "this is a test!") # => 96.55
FuzzyMatch.partial_ratio("world", "hello world") # => 100.0
FuzzyMatch.token_sort_ratio("fuzzy wuzzy was a bear", "wuzzy fuzzy was a bear") # => 100.0
FuzzyMatch.token_set_ratio("fuzzy was a bear", "fuzzy fuzzy was a bear") # => 100.0
FuzzyMatch.extract_one("new york jets", teams).try &.choice # => "New York Jets"
See FuzzyMatch.ratio, FuzzyMatch.partial_ratio, FuzzyMatch.token_sort_ratio,
FuzzyMatch.token_set_ratio, FuzzyMatch.weighted_ratio and FuzzyMatch.extract.
Defined in:
fuzzy_match.crfuzzy_match/extract.cr
fuzzy_match/normalize.cr
fuzzy_match/scorers.cr
Constant Summary
-
VERSION =
"0.1.1"
Class Method Summary
-
.extract(query : String, choices : Enumerable(String), scorer : Scorer = :weighted_ratio, limit : Int32 | Nil = nil, cutoff : Float64 = 0.0) : Array(Match)
Scores query against every string in choices and returns the matches sorted best first.
-
.extract(query : String, choices : Enumerable(String), limit : Int32 | Nil = nil, cutoff : Float64 = 0.0, & : String, String -> Float64) : Array(Match)
Same as
.extract, but scores each pair with the block instead of a built-in scorer. -
.extract_one(query : String, choices : Enumerable(String), scorer : Scorer = :weighted_ratio, cutoff : Float64 = 0.0) : Match | Nil
The best match for query in choices, or
nilwhen nothing reaches cutoff. -
.extract_one(query : String, choices : Enumerable(String), cutoff : Float64 = 0.0, & : String, String -> Float64) : Match | Nil
The best match for query in choices, or
nilwhen nothing reaches cutoff. -
.normalize(s : String) : String
Lowercases s, replaces every run of non-alphanumeric characters with a single space and trims the ends.
-
.partial_ratio(a : String, b : String, normalize : Bool = false) : Float64
Best
.ratioof the shorter string against any window of the longer one. -
.partial_token_set_ratio(a : String, b : String, normalize : Bool = true) : Float64
Like
.token_set_ratio, but compares the leftovers with.partial_ratio. -
.partial_token_sort_ratio(a : String, b : String, normalize : Bool = true) : Float64
Like
.token_sort_ratio, but compares with.partial_ratioso a shorter string whose words all appear in the longer one scores100.0. -
.ratio(a : String, b : String, normalize : Bool = false) : Float64
Plain similarity of the two strings as a whole.
-
.token_set_ratio(a : String, b : String, normalize : Bool = true) : Float64
Similarity based on the sets of words in each string.
-
.token_sort_ratio(a : String, b : String, normalize : Bool = true) : Float64
.ratioof the two strings with their words sorted alphabetically, so word order no longer matters. -
.weighted_ratio(a : String, b : String, normalize : Bool = true) : Float64
The all-rounder: picks the scorer that fits the two strings.
Class Method Detail
Scores query against every string in choices and returns the matches sorted best first. Ties keep the order of choices.
- scorer picks the scoring function;
.weighted_ratioby default. - limit caps the number of matches returned.
- cutoff drops every match scoring below it.
teams = ["Atlanta Falcons", "New York Jets", "New York Giants", "Dallas Cowboys"]
FuzzyMatch.extract("new york jets", teams, limit: 2)
# => [Match(@choice="New York Jets", @score=100.0, @index=1),
# Match(@choice="New York Giants", @score=78.57, @index=2)]
FuzzyMatch.extract("cowboys", teams, scorer: :ratio, cutoff: 90.0) # => []
Same as .extract, but scores each pair with the block instead of a
built-in scorer.
FuzzyMatch.extract("crystal", langs) { |a, b| FuzzyMatch.ratio(a, b, normalize: true) }
The best match for query in choices, or nil when nothing reaches
cutoff. Takes the same arguments as .extract.
FuzzyMatch.extract_one("dallas", teams).try &.choice # => "Dallas Cowboys"
FuzzyMatch.extract_one("vikings", teams, cutoff: 80.0) # => nil
The best match for query in choices, or nil when nothing reaches
cutoff. Takes the same arguments as .extract.
FuzzyMatch.extract_one("dallas", teams).try &.choice # => "Dallas Cowboys"
FuzzyMatch.extract_one("vikings", teams, cutoff: 80.0) # => nil
Lowercases s, replaces every run of non-alphanumeric characters with a
single space and trims the ends. This is what the token scorers and
.weighted_ratio do to their input before comparing.
FuzzyMatch.normalize(" Hello, World! Øl & 123 ") # => "hello world øl 123"
Alphanumeric is decided per Unicode character, so letters outside ASCII (æ, ø, å, ü, ...) are kept and only punctuation and symbols become spaces.
Best .ratio of the shorter string against any window of the longer one.
Useful when one string is a fragment of the other: a surname against a full name, a word against a sentence. The windows include ones that hang over the start or end of the longer string, so a partially matching prefix or suffix still counts.
FuzzyMatch.partial_ratio("world", "hello world") # => 100.0
FuzzyMatch.partial_ratio("Lillevang", "Lillevang Consult ApS") # => 100.0
FuzzyMatch.partial_ratio("kitten", "sitting") # => 66.67
Like .token_set_ratio, but compares the leftovers with .partial_ratio.
Any word in common scores 100.0.
Like .token_sort_ratio, but compares with .partial_ratio so a shorter
string whose words all appear in the longer one scores 100.0.
Plain similarity of the two strings as a whole.
Defined as 2 * LCS / (|a| + |b|) * 100, where LCS is the length of the
longest common subsequence. Equivalently, it is the normalized indel
distance: how many insertions and deletions turn one string into the
other, relative to the combined length. Two empty strings score 100.0.
FuzzyMatch.ratio("this is a test", "this is a test!") # => 96.55
FuzzyMatch.ratio("kitten", "sitting") # => 61.54
FuzzyMatch.ratio("Crystal", "crystal") # => 85.71
FuzzyMatch.ratio("Crystal", "crystal", normalize: true) # => 100.0
Similarity based on the sets of words in each string.
The words both strings share are split off into an intersection, and the
leftovers of each string are compared with and without that common
prefix. Duplicated words and word order are ignored, and a string whose
words are all contained in the other scores 100.0. This is the scorer
for "is this the same thing, written differently?".
FuzzyMatch.token_set_ratio("fuzzy was a bear", "fuzzy fuzzy was a bear") # => 100.0
FuzzyMatch.token_set_ratio("mariners vs angels",
"los angeles angels of anaheim at seattle mariners") # => 90.91
Returns 0.0 when either string has no words at all.
.ratio of the two strings with their words sorted alphabetically, so
word order no longer matters.
FuzzyMatch.token_sort_ratio("fuzzy wuzzy was a bear", "wuzzy fuzzy was a bear") # => 100.0
FuzzyMatch.token_sort_ratio("New York Mets vs Atlanta Braves",
"Atlanta Braves vs New York Mets") # => 100.0
The all-rounder: picks the scorer that fits the two strings.
Strings of similar length are compared with .ratio and the token
scorers. When one string is much longer than the other, the partial
scorers are used instead, and their scores are scaled down so that an
exact match still ranks above a fragment match. The input is normalized
first. This is the default scorer for .extract.
FuzzyMatch.weighted_ratio("New York Jets", "new york jets") # => 100.0
FuzzyMatch.weighted_ratio("Jets", "New York Jets") # => 90.0
FuzzyMatch.weighted_ratio("kitten", "sitting") # => 61.54
Returns 0.0 when either string is empty after normalization.