[Python-Dev] should sre.Scanner be exposed through re and documented?

On Friday 25 April 2003 12:38 pm, Guido van Rossum wrote: > > While moving tests from test_sre to test_re I stumbled upon a simple test > > for sre.Scanner. This looks fairly cool. Should it be exposed through > > re and documented? > > What's Scanner? > You create a Scanner instance with a list of re's and associated functions, then you use it to scan a string, returning a list of parts which match the given re's. (Actually the matches are run through the associated functions, and their output is what forms the returned list.) Here's the single test case Skip refereed to: def s_ident(scanner, token): return token def s_operator(scanner, token): return "op%s" % token def s_float(scanner, token): return float(token) def s_int(scanner, token): return int(token) scanner = sre.Scanner([ (r"[a-zA-Z_]\w*", s_ident), (r"\d+\.\d*", s_float), (r"\d+", s_int), (r"=|\+|-|\*|/", s_operator), (r"\s+", None), ]) # sanity check test('scanner.scan("sum = 3*foo + 312.50 + bar")', (['sum', 'op=', 3, 'op*', 'foo', 'op+', 312.5, 'op+', 'bar'], '')) Gary Herron