haskell-lexer-1.0: A fully compliant Haskell 98 lexer. Source code Contents Index

Language.Haskell.Lexer

Synopsis

type PosToken = (Token, (Pos, String)) data Token = Varid | Conid | Varsym | Consym | Reservedid | Reservedop | Specialid | IntLit | FloatLit | CharLit | StringLit | Qvarid | Qconid | Qvarsym | Qconsym | Special | Whitespace | NestedCommentStart | NestedComment | LiterateComment | Commentstart | Comment | ErrorToken | GotEOF | TheRest | ModuleName | ModuleAlias | Layout | Indent Int | Open Int lexerPass0 :: String -> [PosToken] lexerPass0' :: Pos -> String -> [PosToken] lexerPass1 :: String -> [PosToken] rmSpace :: [PosToken] -> [PosToken] layoutPre :: [PosToken] -> [PosToken] data Pos = Pos { char :: !Int line :: !Int column :: !Int } simpPos :: Pos -> (Int, Int) startPos :: Pos nextPos :: Pos -> String -> Pos nextPos1 :: Pos -> Char -> Pos

Documentation

type PosToken = (Token, (Pos, String)) Source

data Token Source

Haskell token classifications: Constructors Varid Variable Conid Constructor Varsym Variable operator Consym Constructor operator Reservedid Reserved keyword Reservedop Reserved operator Specialid IntLit Integral numeric literal FloatLit Fractional numeric literal CharLit Character literal StringLit String literal Qvarid Qualified variable Qconid Qualified constructor Qvarsym Qualified variable operator Qconsym Qualified constructor operator Special Whitespace White space NestedCommentStart Internal: causes a call to an external function NestedComment A nested comment ({- ... -}) LiterateComment Not handled by the lexer Commentstart Dashes Comment The stuff after the dashes ErrorToken GotEOF TheRest ModuleName ModuleAlias recognized in a later pass Layout for implicit braces Indent Int <n>, to preceed first token on each line Open Int {n}, after let, where, do or of, if not followed by a "{"

lexerPass0 :: String -> [PosToken] Source

Tokenize and add position information. Preserves white space, and does not insert extra tokens due to layout.

lexerPass0' :: Pos -> String -> [PosToken] Source

Same as lexerPass0, except that it uses the given start position.

lexerPass1 :: String -> [PosToken] Source

The function lexerPass1 handles the part of lexical analysis that can be done independently of the parser---the tokenization and the addition of the extra layout tokens <n> and {n}, as specified in section 9.3 of the revised Haskell 98 Report.

rmSpace :: [PosToken] -> [PosToken] Source

Remove token that are not meaningful (e.g., white space and comments).

layoutPre :: [PosToken] -> [PosToken] Source

This is an implementation of Haskell layout, as specified in section 9.3 of the revised Haskell 98 report. This preprocessor inserts the extra <n> and {n} tokens.

data Pos Source

The posisiotn within a file. Constructors Pos char :: !Int line :: !Int column :: !Int

simpPos :: Pos -> (Int, Int) Source

The line and column numbers of a position.

startPos :: Pos Source

The first column is designated column 1, not 0.

nextPos :: Pos -> String -> Pos Source

Advance position by a string.

nextPos1 :: Pos -> Char -> Pos Source

Advance position by a single character.

Produced by Haddock version 2.6.0