Module lexbase

This module implements a base object of a lexer with efficient buffer handling. Only at line endings checks are necessary if the buffer needs refilling.

Types

BaseLexer = object of RootObj
  bufpos*: int                ## the current position within the buffer
  buf*: cstring               ## the buffer itself
  bufLen*: int                ## length of buffer in characters
  input: Stream               ## the input stream
  lineNumber*: int            ## the current line number
  sentinel: int
  lineStart: int
  fileOpened: bool
the base lexer. Inherit your lexer from this object.   Source

Consts

EndOfFile = '\0'
end of file marker   Source
NewLines = {'\x0D', '\x0A'}
  Source

Procs

proc close(L: var BaseLexer) {.raises: [Exception], tags: [].}
closes the base lexer. This closes L's associated stream too.   Source
proc handleCR(L: var BaseLexer; pos: int): int {.raises: [Exception], 
    tags: [ReadIOEffect].}
Call this if you scanned over 'c' in the buffer; it returns the the position to continue the scanning from. pos must be the position of the 'c'.   Source
proc handleLF(L: var BaseLexer; pos: int): int {.raises: [Exception], 
    tags: [ReadIOEffect].}
Call this if you scanned over 'L' in the buffer; it returns the the position to continue the scanning from. pos must be the position of the 'L'.   Source
proc open(L: var BaseLexer; input: Stream; bufLen: int = 8192) {.
    raises: [Exception], tags: [ReadIOEffect].}
inits the TBaseLexer with a stream to read from   Source
proc getColNumber(L: BaseLexer; pos: int): int {.raises: [], tags: [].}
retrieves the current column.   Source
proc getCurrentLine(L: BaseLexer; marker: bool = true): string {.raises: [], 
    tags: [].}
retrieves the current line.   Source