ByoungSooPark/modeling-app
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add the ability to recast comments and some whitespace (#10)

Browse Source 
* Add the ability to recast comments and some whitespace

Currently because whitespace or anything that's not needed for execution is not stored in the AST, it's hard to respect things like user formatting when recasting.

I think having a by-default-opinioned formatter is a good thing, but where this becomes problematic is when users wants to simply leave a blank space between some lines for a bit of breathing room, a code paragraph if you will, but maybe more importantly comments have not been implemented for the same reason, there wasn't a way with the current setup to insert them back in.

In some ways the most straightforward way to do this is to put whitespace and comments into the AST. Even though they are not crucial for execution, code-gen/recasting needs to be a first-class citizen in this lang so that's probably the long-term solution. However I'm trying to draw inspiration from other languages, and since it's not the norm to put comments et-al into the AST I haven't done so.

Because whitespace is tokenised already if not transformed into the AST, there is somewhat of a map of these things without going back to source code, so atm I'm experimenting with using this to insert extra linebreaks and comments back in between statements. I think this is a good compromise for the time being for what is a nice to have feature atm.

Because it's only going to respect non-code parts in between statements this will mean that you can't format objects or function params how you like (but I think this is good to have an opinioned fmt out of the box) and comments like myFunctionCall('a', /* inline comment */ b) will not work either.

* clean up
This commit is contained in:
Kurt Hutten
2023-01-23 14:50:58 +11:00
committed by GitHub
parent 18629ea50c
commit dd5022b38e
9 changed files with 720 additions and 58 deletions
Download Patch File Download Diff File Expand all files Collapse all files

27
src/lang/abstractSyntaxTree.ts
View File

@ -83,7 +83,7 @@ export interface Program {
type: syntaxType
start: number
end: number
body: Body[]
body: BodyItem[]
}
interface GeneralStatement {
type: syntaxType
@ -937,7 +937,7 @@ function makeParams(
export interface BlockStatement extends GeneralStatement {
type: 'BlockStatement'
body: Body[]
body: BodyItem[]
}
function makeBlockStatement(
@ -996,7 +996,7 @@ function nextMeaningfulToken(
if (!token) {
return { token, index: tokens.length }
}
if (token.type === 'whitespace') {
if (isNotCodeToken(token)) {
return nextMeaningfulToken(tokens, index, offset + 1)
}
return { token, index: newIndex }
@ -1012,13 +1012,16 @@ function previousMeaningfulToken(
if (!token) {
return { token, index: 0 }
}
if (token.type === 'whitespace') {
if (isNotCodeToken(token)) {
return previousMeaningfulToken(tokens, index, offset + 1)
}
return { token, index: newIndex }
}
type Body = ExpressionStatement | VariableDeclaration | ReturnStatement
export type BodyItem =
| ExpressionStatement
| VariableDeclaration
| ReturnStatement
function makeBody(
{
@ -1028,8 +1031,8 @@ function makeBody(
tokens: Token[]
tokenIndex?: number
},
previousBody: Body[] = []
): { body: Body[]; lastIndex: number } {
previousBody: BodyItem[] = []
): { body: BodyItem[]; lastIndex: number } {
if (tokenIndex >= tokens.length) {
return { body: previousBody, lastIndex: tokenIndex }
}
@ -1041,7 +1044,7 @@ function makeBody(
if (typeof token === 'undefined') {
console.log('probably should throw')
}
if (token.type === 'whitespace') {
if (isNotCodeToken(token)) {
return makeBody({ tokens, tokenIndex: tokenIndex + 1 }, previousBody)
}
const nextToken = nextMeaningfulToken(tokens, tokenIndex)
@ -1532,3 +1535,11 @@ export function getNodePathFromSourceRange(
}
return path
}
export function isNotCodeToken(token: Token): boolean {
return (
token.type === 'whitespace' ||
token.type === 'linecomment' ||
token.type === 'blockcomment'
)
}