clang  3.7.0
Public Member Functions | Public Attributes | List of all members
clang::format::FormatToken Struct Reference

A wrapper around a Token storing information about the whitespace characters preceding it. More...

#include <FormatToken.h>

Collaboration diagram for clang::format::FormatToken:
[legend]

Public Member Functions

 FormatToken ()
 
bool is (tok::TokenKind Kind) const
 
bool is (TokenType TT) const
 
bool is (const IdentifierInfo *II) const
 
template<typename A , typename B >
bool isOneOf (A K1, B K2) const
 
template<typename A , typename B , typename... Ts>
bool isOneOf (A K1, B K2, Ts...Ks) const
 
template<typename T >
bool isNot (T Kind) const
 
bool isStringLiteral () const
 
bool isObjCAtKeyword (tok::ObjCKeywordKind Kind) const
 
bool isAccessSpecifier (bool ColonRequired=true) const
 
bool isSimpleTypeSpecifier () const
 Determine whether the token is a simple-type-specifier. More...
 
bool isObjCAccessSpecifier () const
 
bool opensScope () const
 Returns whether Tok is ([{ or a template opening <. More...
 
bool closesScope () const
 Returns whether Tok is )]} or a template closing >. More...
 
bool isMemberAccess () const
 Returns true if this is a "." or "->" accessing a member. More...
 
bool isUnaryOperator () const
 
bool isBinaryOperator () const
 
bool isTrailingComment () const
 
bool isFunctionLikeKeyword () const
 Returns true if this is a keyword that can be used like a function call (e.g. sizeof, typeid, ...). More...
 
SourceLocation getStartOfNonWhitespace () const
 Returns actual token start location without leading escaped newlines and whitespace. More...
 
prec::Level getPrecedence () const
 
FormatTokengetPreviousNonComment () const
 Returns the previous token ignoring comments. More...
 
const FormatTokengetNextNonComment () const
 Returns the next token ignoring comments. More...
 
bool opensBlockTypeList (const FormatStyle &Style) const
 Returns true if this tokens starts a block-type list, i.e. a list that should be indented with a block indent. More...
 
bool closesBlockTypeList (const FormatStyle &Style) const
 Same as opensBlockTypeList, but for the closing token. More...
 

Public Attributes

Token Tok
 The Token. More...
 
unsigned NewlinesBefore = 0
 The number of newlines immediately before the Token. More...
 
bool HasUnescapedNewline = false
 Whether there is at least one unescaped newline before the Token. More...
 
SourceRange WhitespaceRange
 The range of the whitespace immediately preceding the Token. More...
 
unsigned LastNewlineOffset = 0
 The offset just past the last '
' in this token's leading whitespace (relative to WhiteSpaceStart). 0 if there is no '
'. More...
 
unsigned ColumnWidth = 0
 The width of the non-whitespace parts of the token (or its first line for multi-line tokens) in columns. We need this to correctly measure number of columns a token spans. More...
 
unsigned LastLineColumnWidth = 0
 Contains the width in columns of the last line of a multi-line token. More...
 
bool IsMultiline = false
 Whether the token text contains newlines (escaped or not). More...
 
bool IsFirst = false
 Indicates that this is the first token. More...
 
bool MustBreakBefore = false
 Whether there must be a line break before this token. More...
 
StringRef TokenText
 The raw text of the token. More...
 
bool IsUnterminatedLiteral = 0
 Set to true if this token is an unterminated literal. More...
 
BraceBlockKind BlockKind = BK_Unknown
 Contains the kind of block if this token is a brace. More...
 
TokenType Type = TT_Unknown
 
unsigned SpacesRequiredBefore = 0
 The number of spaces that should be inserted before this token. More...
 
bool CanBreakBefore = false
 true if it is allowed to break before this token. More...
 
bool ClosesTemplateDeclaration = false
 true if this is the ">" of "template<..>". More...
 
unsigned ParameterCount = 0
 Number of parameters, if this is "(", "[" or "<". More...
 
unsigned BlockParameterCount = 0
 Number of parameters that are nested blocks, if this is "(", "[" or "<". More...
 
tok::TokenKind ParentBracket = tok::unknown
 If this is a bracket ("<", "(", "[" or "{"), contains the kind of the surrounding bracket. More...
 
std::unique_ptr< TokenRoleRole
 A token can have a special role that can carry extra information about the token's formatting. More...
 
ParameterPackingKind PackingKind = PPK_Inconclusive
 If this is an opening parenthesis, how are the parameters packed? More...
 
unsigned TotalLength = 0
 The total length of the unwrapped line up to and including this token. More...
 
unsigned OriginalColumn = 0
 The original 0-based column of this token, including expanded tabs. The configured TabWidth is used as tab width. More...
 
unsigned UnbreakableTailLength = 0
 The length of following tokens until the next natural split point, or the next token that can be broken. More...
 
unsigned BindingStrength = 0
 The binding strength of a token. This is a combined value of operator precedence, parenthesis nesting, etc. More...
 
unsigned NestingLevel = 0
 The nesting level of this token, i.e. the number of surrounding (), [], {} or <>. More...
 
unsigned SplitPenalty = 0
 Penalty for inserting a line break before this token. More...
 
unsigned LongestObjCSelectorName = 0
 If this is the first ObjC selector name in an ObjC method definition or call, this contains the length of the longest name. More...
 
SmallVector< prec::Level, 4 > FakeLParens
 Stores the number of required fake parentheses and the corresponding operator precedence. More...
 
unsigned FakeRParens = 0
 Insert this many fake ) after this token for correct indentation. More...
 
bool StartsBinaryExpression = false
 true if this token starts a binary expression, i.e. has at least one fake l_paren with a precedence greater than prec::Unknown. More...
 
bool EndsBinaryExpression = false
 true if this token ends a binary expression. More...
 
unsigned OperatorIndex = 0
 Is this is an operator (or "."/"->") in a sequence of operators with the same precedence, contains the 0-based operator index. More...
 
bool LastOperator = false
 Is this the last operator (or "."/"->") in a sequence of operators with the same precedence? More...
 
bool PartOfMultiVariableDeclStmt = false
 Is this token part of a DeclStmt defining multiple variables? More...
 
FormatTokenMatchingParen = nullptr
 If this is a bracket, this points to the matching one. More...
 
FormatTokenPrevious = nullptr
 The previous token in the unwrapped line. More...
 
FormatTokenNext = nullptr
 The next token in the unwrapped line. More...
 
SmallVector< AnnotatedLine *, 1 > Children
 If this token starts a block, this contains all the unwrapped lines in it. More...
 
FormatDecision Decision = FD_Unformatted
 Stores the formatting decision for the token once it was made. More...
 
bool Finalized = false
 If true, this token has been fully formatted (indented and potentially re-formatted inside), and we do not allow further formatting changes. More...
 

Detailed Description

A wrapper around a Token storing information about the whitespace characters preceding it.

Definition at line 112 of file FormatToken.h.

Constructor & Destructor Documentation

clang::format::FormatToken::FormatToken ( )
inline

Definition at line 113 of file FormatToken.h.

Member Function Documentation

bool clang::format::FormatToken::closesBlockTypeList ( const FormatStyle Style) const
inline

Same as opensBlockTypeList, but for the closing token.

Definition at line 419 of file FormatToken.h.

References MatchingParen, and opensBlockTypeList().

Referenced by clang::format::ContinuationIndenter::canBreak(), and clang::format::ContinuationIndenter::mustBreak().

bool clang::format::FormatToken::closesScope ( ) const
inline

Returns whether Tok is )]} or a template closing >.

Definition at line 322 of file FormatToken.h.

References isOneOf().

Referenced by clang::format::startsSegmentOfBuilderTypeCall().

const FormatToken* clang::format::FormatToken::getNextNonComment ( ) const
inline

Returns the next token ignoring comments.

Definition at line 402 of file FormatToken.h.

References is(), Next, and Tok.

Referenced by clang::format::CommaSeparatedList::precomputeFormattingInfos().

prec::Level clang::format::FormatToken::getPrecedence ( ) const
inline
FormatToken* clang::format::FormatToken::getPreviousNonComment ( ) const
inline

Returns the previous token ignoring comments.

Definition at line 394 of file FormatToken.h.

References is(), Previous, and Tok.

Referenced by clang::format::CommaSeparatedList::formatAfterToken(), and clang::format::CommaSeparatedList::precomputeFormattingInfos().

SourceLocation clang::format::FormatToken::getStartOfNonWhitespace ( ) const
inline

Returns actual token start location without leading escaped newlines and whitespace.

This can be different to Tok.getLocation(), which includes leading escaped newlines.

Definition at line 385 of file FormatToken.h.

References clang::SourceRange::getEnd(), and WhitespaceRange.

Referenced by clang::format::WhitespaceManager::replaceWhitespaceInToken().

bool clang::format::FormatToken::is ( tok::TokenKind  Kind) const
inline
bool clang::format::FormatToken::is ( TokenType  TT) const
inline

Definition at line 282 of file FormatToken.h.

bool clang::format::FormatToken::is ( const IdentifierInfo II) const
inline

Definition at line 283 of file FormatToken.h.

References clang::Token::getIdentifierInfo(), and Tok.

bool clang::format::FormatToken::isAccessSpecifier ( bool  ColonRequired = true) const
inline

Definition at line 301 of file FormatToken.h.

References is(), isOneOf(), and Next.

bool clang::format::FormatToken::isBinaryOperator ( ) const
inline

Definition at line 350 of file FormatToken.h.

References clang::prec::Comma, and getPrecedence().

bool clang::format::FormatToken::isFunctionLikeKeyword ( ) const
inline

Returns true if this is a keyword that can be used like a function call (e.g. sizeof, typeid, ...).

Definition at line 362 of file FormatToken.h.

References clang::Token::getKind(), and Tok.

bool clang::format::FormatToken::isMemberAccess ( ) const
inline

Returns true if this is a "." or "->" accessing a member.

Definition at line 328 of file FormatToken.h.

References isOneOf().

Referenced by clang::format::ContinuationIndenter::canBreak(), and clang::format::startsSegmentOfBuilderTypeCall().

template<typename T >
bool clang::format::FormatToken::isNot ( Kind) const
inline
bool clang::format::FormatToken::isObjCAccessSpecifier ( ) const
inline

Definition at line 309 of file FormatToken.h.

References is(), isObjCAtKeyword(), and Next.

bool clang::format::FormatToken::isObjCAtKeyword ( tok::ObjCKeywordKind  Kind) const
inline

Definition at line 297 of file FormatToken.h.

References clang::Token::isObjCAtKeyword(), and Tok.

Referenced by isObjCAccessSpecifier().

template<typename A , typename B >
bool clang::format::FormatToken::isOneOf ( K1,
K2 
) const
inline
template<typename A , typename B , typename... Ts>
bool clang::format::FormatToken::isOneOf ( K1,
K2,
Ts...  Ks 
) const
inline

Definition at line 290 of file FormatToken.h.

References is(), and isOneOf().

bool clang::format::FormatToken::isSimpleTypeSpecifier ( ) const

Determine whether the token is a simple-type-specifier.

Definition at line 42 of file FormatToken.cpp.

References clang::Token::getKind(), and Tok.

bool clang::format::FormatToken::isStringLiteral ( ) const
inline

Definition at line 295 of file FormatToken.h.

References clang::Token::getKind(), clang::tok::isStringLiteral(), and Tok.

bool clang::format::FormatToken::isTrailingComment ( ) const
inline
bool clang::format::FormatToken::isUnaryOperator ( ) const
inline

Definition at line 334 of file FormatToken.h.

References clang::Token::getKind(), and Tok.

bool clang::format::FormatToken::opensBlockTypeList ( const FormatStyle Style) const
inline

Returns true if this tokens starts a block-type list, i.e. a list that should be indented with a block indent.

Definition at line 411 of file FormatToken.h.

References clang::format::BK_Block, BlockKind, clang::format::FormatStyle::Cpp11BracedListStyle, is(), and NestingLevel.

Referenced by closesBlockTypeList().

bool clang::format::FormatToken::opensScope ( ) const
inline

Returns whether Tok is ([{ or a template opening <.

Definition at line 317 of file FormatToken.h.

References isOneOf().

Referenced by clang::format::TokenAnnotator::calculateFormattingInformation(), and clang::format::ContinuationIndenter::canBreak().

Member Data Documentation

unsigned clang::format::FormatToken::BindingStrength = 0

The binding strength of a token. This is a combined value of operator precedence, parenthesis nesting, etc.

Definition at line 216 of file FormatToken.h.

Referenced by clang::format::TokenAnnotator::calculateFormattingInformation().

BraceBlockKind clang::format::FormatToken::BlockKind = BK_Unknown
unsigned clang::format::FormatToken::BlockParameterCount = 0

Number of parameters that are nested blocks, if this is "(", "[" or "<".

Definition at line 188 of file FormatToken.h.

bool clang::format::FormatToken::CanBreakBefore = false
SmallVector<AnnotatedLine *, 1> clang::format::FormatToken::Children

If this token starts a block, this contains all the unwrapped lines in it.

Definition at line 271 of file FormatToken.h.

Referenced by clang::format::AnnotatedLine::AnnotatedLine(), clang::format::TokenAnnotator::calculateFormattingInformation(), and clang::format::AnnotatedLine::~AnnotatedLine().

bool clang::format::FormatToken::ClosesTemplateDeclaration = false

true if this is the ">" of "template<..>".

Definition at line 177 of file FormatToken.h.

Referenced by clang::format::ContinuationIndenter::mustBreak().

unsigned clang::format::FormatToken::ColumnWidth = 0

The width of the non-whitespace parts of the token (or its first line for multi-line tokens) in columns. We need this to correctly measure number of columns a token spans.

Definition at line 138 of file FormatToken.h.

Referenced by clang::format::TokenAnnotator::calculateFormattingInformation(), clang::format::CodePointsBetween(), and clang::format::CommaSeparatedList::formatAfterToken().

FormatDecision clang::format::FormatToken::Decision = FD_Unformatted

Stores the formatting decision for the token once it was made.

Definition at line 274 of file FormatToken.h.

Referenced by clang::format::WhitespaceManager::replaceWhitespace().

bool clang::format::FormatToken::EndsBinaryExpression = false

true if this token ends a binary expression.

Definition at line 245 of file FormatToken.h.

Referenced by clang::format::ContinuationIndenter::mustBreak().

SmallVector<prec::Level, 4> clang::format::FormatToken::FakeLParens

Stores the number of required fake parentheses and the corresponding operator precedence.

If multiple fake parentheses start at a token, this vector stores them in reverse order, i.e. inner fake parenthesis first.

Definition at line 237 of file FormatToken.h.

unsigned clang::format::FormatToken::FakeRParens = 0

Insert this many fake ) after this token for correct indentation.

Definition at line 239 of file FormatToken.h.

bool clang::format::FormatToken::Finalized = false

If true, this token has been fully formatted (indented and potentially re-formatted inside), and we do not allow further formatting changes.

Definition at line 279 of file FormatToken.h.

Referenced by clang::format::WhitespaceManager::addUntouchableToken(), clang::format::WhitespaceManager::replaceWhitespace(), and clang::format::WhitespaceManager::replaceWhitespaceInToken().

bool clang::format::FormatToken::HasUnescapedNewline = false

Whether there is at least one unescaped newline before the Token.

Definition at line 126 of file FormatToken.h.

Referenced by clang::format::TokenAnnotator::calculateFormattingInformation(), and clang::format::CommaSeparatedList::precomputeFormattingInfos().

bool clang::format::FormatToken::IsFirst = false
bool clang::format::FormatToken::IsMultiline = false

Whether the token text contains newlines (escaped or not).

Definition at line 145 of file FormatToken.h.

Referenced by clang::format::TokenAnnotator::calculateFormattingInformation().

bool clang::format::FormatToken::IsUnterminatedLiteral = 0

Set to true if this token is an unterminated literal.

Definition at line 163 of file FormatToken.h.

unsigned clang::format::FormatToken::LastLineColumnWidth = 0

Contains the width in columns of the last line of a multi-line token.

Definition at line 142 of file FormatToken.h.

unsigned clang::format::FormatToken::LastNewlineOffset = 0

The offset just past the last '
' in this token's leading whitespace (relative to WhiteSpaceStart). 0 if there is no '
'.

Definition at line 133 of file FormatToken.h.

Referenced by clang::format::ContinuationIndenter::addTokenToState().

bool clang::format::FormatToken::LastOperator = false

Is this the last operator (or "."/"->") in a sequence of operators with the same precedence?

Definition at line 253 of file FormatToken.h.

unsigned clang::format::FormatToken::LongestObjCSelectorName = 0

If this is the first ObjC selector name in an ObjC method definition or call, this contains the length of the longest name.

This being set to 0 means that the selectors should not be colon-aligned, e.g. because several of them are block-type.

Definition at line 230 of file FormatToken.h.

FormatToken* clang::format::FormatToken::MatchingParen = nullptr
bool clang::format::FormatToken::MustBreakBefore = false
unsigned clang::format::FormatToken::NestingLevel = 0
unsigned clang::format::FormatToken::NewlinesBefore = 0

The number of newlines immediately before the Token.

This can be used to determine what the user wrote in the original code and thereby e.g. leave an empty line between two function definitions.

Definition at line 122 of file FormatToken.h.

Referenced by clang::format::WhitespaceManager::addUntouchableToken(), clang::format::UnwrappedLineFormatter::format(), and isTrailingComment().

FormatToken* clang::format::FormatToken::Next = nullptr
unsigned clang::format::FormatToken::OperatorIndex = 0

Is this is an operator (or "."/"->") in a sequence of operators with the same precedence, contains the 0-based operator index.

Definition at line 249 of file FormatToken.h.

unsigned clang::format::FormatToken::OriginalColumn = 0

The original 0-based column of this token, including expanded tabs. The configured TabWidth is used as tab width.

Definition at line 207 of file FormatToken.h.

Referenced by clang::format::WhitespaceManager::addUntouchableToken(), and clang::format::UnwrappedLineFormatter::format().

ParameterPackingKind clang::format::FormatToken::PackingKind = PPK_Inconclusive

If this is an opening parenthesis, how are the parameters packed?

Definition at line 199 of file FormatToken.h.

unsigned clang::format::FormatToken::ParameterCount = 0

Number of parameters, if this is "(", "[" or "<".

This is initialized to 1 as we don't need to distinguish functions with 0 parameters from functions with 1 parameter. Thus, we can simply count the number of commas.

Definition at line 184 of file FormatToken.h.

tok::TokenKind clang::format::FormatToken::ParentBracket = tok::unknown

If this is a bracket ("<", "(", "[" or "{"), contains the kind of the surrounding bracket.

Definition at line 192 of file FormatToken.h.

bool clang::format::FormatToken::PartOfMultiVariableDeclStmt = false

Is this token part of a DeclStmt defining multiple variables?

Only set if Type == TT_StartOfName.

Definition at line 258 of file FormatToken.h.

FormatToken* clang::format::FormatToken::Previous = nullptr
std::unique_ptr<TokenRole> clang::format::FormatToken::Role

A token can have a special role that can carry extra information about the token's formatting.

Definition at line 196 of file FormatToken.h.

Referenced by clang::format::TokenAnnotator::calculateFormattingInformation(), and clang::format::AnnotatedLine::~AnnotatedLine().

unsigned clang::format::FormatToken::SpacesRequiredBefore = 0

The number of spaces that should be inserted before this token.

Definition at line 171 of file FormatToken.h.

Referenced by clang::format::TokenAnnotator::annotate(), and clang::format::TokenAnnotator::calculateFormattingInformation().

unsigned clang::format::FormatToken::SplitPenalty = 0

Penalty for inserting a line break before this token.

Definition at line 223 of file FormatToken.h.

Referenced by clang::format::TokenAnnotator::calculateFormattingInformation().

bool clang::format::FormatToken::StartsBinaryExpression = false

true if this token starts a binary expression, i.e. has at least one fake l_paren with a precedence greater than prec::Unknown.

Definition at line 243 of file FormatToken.h.

Token clang::format::FormatToken::Tok
StringRef clang::format::FormatToken::TokenText
unsigned clang::format::FormatToken::TotalLength = 0
TokenType clang::format::FormatToken::Type = TT_Unknown
unsigned clang::format::FormatToken::UnbreakableTailLength = 0

The length of following tokens until the next natural split point, or the next token that can be broken.

Definition at line 211 of file FormatToken.h.

SourceRange clang::format::FormatToken::WhitespaceRange

The documentation for this struct was generated from the following files: