Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added support for atomic vectors #1245

Draft
wants to merge 27 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
765aa6b
feat(vector-def): add processVector
Slartibartfass2 Jan 15, 2025
ba938a3
refactor: invert pointerTracking config check
Slartibartfass2 Jan 15, 2025
fddc035
feat-fix: remove pointerTracking check for remove child edges block
Slartibartfass2 Jan 15, 2025
f0b5395
feat: add index identifer with index instead of lexeme
Slartibartfass2 Jan 16, 2025
14773a0
feat(list-def): store index of list arguments
Slartibartfass2 Jan 16, 2025
78bcb59
feat(vector-def): add flattened indices for nested vectors
Slartibartfass2 Jan 16, 2025
3444e3a
test(vector-def): add tests for vector definitions
Slartibartfass2 Jan 17, 2025
ba8c099
refactor(vector-def): extract flatten indices logic to separate method
Slartibartfass2 Jan 17, 2025
8eee205
feat-fix(vector-def): also support logical and string values
Slartibartfass2 Jan 17, 2025
b41f8c5
refactor(built-in-access): extract method to resolve indices
Slartibartfass2 Jan 17, 2025
0418df9
refactor: rename pointer analysis test files
Slartibartfass2 Jan 17, 2025
d824be6
feat: add support for single index-access
Slartibartfass2 Jan 17, 2025
316f5b9
refactor: extract access operand extraction
Slartibartfass2 Jan 17, 2025
8ac632c
refactor: extract construct accessed indices logic for replacement func
Slartibartfass2 Jan 17, 2025
2f6f7bf
test: add tests for list definitions
Slartibartfass2 Jan 20, 2025
8b2e251
feat: process indices on the way up the fold
Slartibartfass2 Jan 20, 2025
8c221f7
refactor: rename list-access.ts utils to containers.ts
Slartibartfass2 Jan 20, 2025
e7bd400
refactor: extract resolve indices by argument name logic
Slartibartfass2 Jan 20, 2025
07c6ae4
test(container-def): add tests for passing nested container as variable
Slartibartfass2 Jan 20, 2025
e6d1907
test(vector-access): make tests also for single bracket access
Slartibartfass2 Jan 21, 2025
101f7bb
feat: add support for numerical index based assignment
Slartibartfass2 Jan 21, 2025
9d2e198
test: add dataflow tests for access and assignment
Slartibartfass2 Jan 21, 2025
e38bf80
test: make single index based access tests parameterized
Slartibartfass2 Jan 21, 2025
21c22ca
test: add ability to create graph edges with queries
Slartibartfass2 Jan 25, 2025
10f5276
test-fix: make tests work again after rebase
Slartibartfass2 Jan 25, 2025
ef2bdc1
feat: add support for container assignment
Slartibartfass2 Jan 25, 2025
b4f8b0a
test: add tests for unknown container definition
Slartibartfass2 Jan 25, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions src/dataflow/environments/built-in.ts
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ import { registerBuiltInDefinitions } from './built-in-config';
import { DefaultBuiltinConfig } from './default-builtin-config';
import type { LinkTo } from '../../queries/catalog/call-context-query/call-context-query-format';
import { processList } from '../internal/process/functions/call/built-in/built-in-list';
import { processVector } from '../internal/process/functions/call/built-in/built-in-vector';



Expand Down Expand Up @@ -149,6 +150,7 @@ export const BuiltInProcessorMapper = {
'builtin:while-loop': processWhileLoop,
'builtin:replacement': processReplacementFunction,
'builtin:list': processList,
'builtin:vector': processVector,
} as const satisfies Record<`builtin:${string}`, BuiltInIdentifierProcessorWithConfig<never>>;

export type BuiltInMappingName = keyof typeof BuiltInProcessorMapper;
Expand Down
3 changes: 2 additions & 1 deletion src/dataflow/environments/default-builtin-config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ export const DefaultBuiltinConfig: BuiltInDefinitions = [
{
type: 'function',
names: [
'c', 't', 'aperm' /* vector construction, concatenation, transpose function, permutation generation */
't', 'aperm' /* transpose function, permutation generation */
],
processor: 'builtin:default',
config: { readAllArguments: true },
Expand Down Expand Up @@ -118,6 +118,7 @@ export const DefaultBuiltinConfig: BuiltInDefinitions = [
{ type: 'function', names: ['while'], processor: 'builtin:while-loop', config: {}, assumePrimitive: true },
{ type: 'function', names: ['do.call'], processor: 'builtin:apply', config: { indexOfFunction: 0, unquoteFunction: true }, assumePrimitive: true },
{ type: 'function', names: ['list'], processor: 'builtin:list', config: {}, assumePrimitive: true },
{ type: 'function', names: ['c'], processor: 'builtin:vector', config: {}, assumePrimitive: true },
{
type: 'function',
names: [
Expand Down
8 changes: 4 additions & 4 deletions src/dataflow/environments/define.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ import type { IEnvironment, REnvironmentInformation } from './environment';
import { cloneEnvironmentInformation } from './clone';
import type { IdentifierDefinition, InGraphIdentifierDefinition } from './identifier';
import type { ContainerIndex, ContainerIndices } from '../graph/vertex';
import { isParentContainerIndex } from '../graph/vertex';
import { isParentContainerIndex, isSameIndex } from '../graph/vertex';


function defInEnv(newEnvironments: IEnvironment, name: string, definition: IdentifierDefinition) {
Expand Down Expand Up @@ -70,7 +70,7 @@ function overwriteContainerIndices(
if(isParentContainerIndex(overwriteIndex)) {
newIndices = [];
for(const index of indices.indices) {
if(index.lexeme === overwriteIndex.lexeme && isParentContainerIndex(index)) {
if(isSameIndex(index, overwriteIndex) && isParentContainerIndex(index)) {
const overwriteSubIndices = overwriteIndex.subIndices.flatMap(a => a.indices);

let newSubIndices: ContainerIndices[] = index.subIndices;
Expand All @@ -85,7 +85,7 @@ function overwriteContainerIndices(
});
}
}
if(index.lexeme !== overwriteIndex.lexeme || !isParentContainerIndex(index)) {
if(!isSameIndex(index, overwriteIndex) || !isParentContainerIndex(index)) {
newIndices.push(index);
}
}
Expand All @@ -94,7 +94,7 @@ function overwriteContainerIndices(
newIndices = indices.indices;
} else {
// Filter existing indices with the same name
newIndices = indices.indices.filter(def => def.lexeme !== overwriteIndex.lexeme);
newIndices = indices.indices.filter(def => !isSameIndex(def, overwriteIndex));
}

if(indices.isContainer || newIndices.length > 0) {
Expand Down
11 changes: 11 additions & 0 deletions src/dataflow/graph/dataflowgraph-builder.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,11 @@ import { EdgeType } from './edge';
import type { ControlDependency } from '../info';
import type { LinkTo } from '../../queries/catalog/call-context-query/call-context-query-format';
import { DefaultBuiltinConfig } from '../environments/default-builtin-config';
import type { FlowrSearchLike } from '../../search/flowr-search-builder';
import { runSearch } from '../../search/flowr-search-executor';
import type { Pipeline } from '../../core/steps/pipeline/pipeline';
import type { FlowrSearchInput } from '../../search/flowr-search';
import { guard } from '../../util/assert';

export function emptyGraph(idMap?: AstIdMap) {
return new DataflowGraphBuilder(idMap);
Expand Down Expand Up @@ -206,6 +211,12 @@ export class DataflowGraphBuilder extends DataflowGraph {
return this.edgeHelper(from, to, EdgeType.Reads);
}

public readsQuery<P extends Pipeline>(from: FlowrSearchLike, to: DataflowGraphEdgeTarget, data: FlowrSearchInput<P>) {
const result = runSearch(from, data);
guard(result.length == 1, 'query result should yield only one node');
return this.reads(result[0].node.info.id, to);
}

/**
* Adds a **defined-by edge** (E2), with from as defined variable, and to
* as a variable/function contributing to its definition.
Expand Down
73 changes: 71 additions & 2 deletions src/dataflow/graph/vertex.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,16 +16,53 @@ export enum VertexType {
export const ValidVertexTypes: Set<string> = new Set(Object.values(VertexType));
export const ValidVertexTypeReverse = Object.fromEntries(Object.entries(VertexType).map(([k, v]) => [v, k]));

/**
* Identifier for arguments e.g. for `3` in `c(2, 3, 5)` the identifier would be
* ```ts
* {
* index: 2
* }
* ```
*/
export interface UnnamedArgumentId {
readonly index: number,
}

/**
* Identifier for named arguments e.g. for `age` in `list(name = 'John', age = 8)`
* the indentifier would be
* ```ts
* {
* index: 2,
* lexeme: 'age'
* }
* ```
*/
export interface NamedArgumentId {
/**
* Index may be undefined, when no index information is available.
*/
readonly index: number | undefined;

readonly lexeme: string,
}

export function isNamedArgumentId(identifier: IndexIdentifier): identifier is NamedArgumentId {
return 'lexeme' in identifier;
}

export type IndexIdentifier = UnnamedArgumentId | NamedArgumentId;

/**
* A single index of a container, which is not a container itself.
*
* This can be e.g. a string, number or boolean index.
*/
export interface ContainerLeafIndex {
/**
* Distinctive lexeme of index e.g. 'name' for `list(name = 'John')`
* Distinctive identifier of index, see {@link IndexIdentifier}.
*/
readonly lexeme: string,
readonly identifier: IndexIdentifier,

/**
* NodeId of index in graph.
Expand Down Expand Up @@ -57,6 +94,38 @@ export function isParentContainerIndex(index: ContainerIndex): index is Containe
*/
export type ContainerIndex = ContainerLeafIndex | ContainerParentIndex;

/**
* Checks whether {@link index} is accessed by {@link accessLexeme}.
*
* @param index - The {@link ContainerIndex}, which is accessed
* @param accessLexeme - The access lexeme
* @param isIndexBasedAccess - Whether the index of the {@link ContainerIndex} is accessed i.e. the position in the container and not e.g. the name of the index
* @returns true, when {@link accessLexeme} accesses the {@link index}, false otherwise
*/
export function isAccessed(index: ContainerIndex, accessLexeme: string, isIndexBasedAccess: boolean) {
if(isIndexBasedAccess) {
return index.identifier.index === Number(accessLexeme);
}

if(isNamedArgumentId(index.identifier)) {
return index.identifier.lexeme === accessLexeme;
}

return false;
}

export function isSameIndex(a: ContainerIndex, b: ContainerIndex) {
if(isNamedArgumentId(a.identifier) && isNamedArgumentId(b.identifier)) {
return a.identifier.lexeme === b.identifier.lexeme;
}

if(a.identifier.index === undefined || b.identifier.index === undefined) {
return false;
}

return a.identifier.index === b.identifier.index;
}

/**
* List of indices of a single statement like `list(a=3, b=2)`
*/
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,7 @@ import { ReferenceType } from '../../../../../environments/identifier';
import type { ContainerIndicesCollection, ContainerParentIndex } from '../../../../../graph/vertex';
import { isParentContainerIndex } from '../../../../../graph/vertex';
import type { RArgument } from '../../../../../../r-bridge/lang-4.x/ast/model/nodes/r-argument';
import { RoleInParent } from '../../../../../../r-bridge/lang-4.x/ast/model/processing/role';
import { filterIndices, resolveSingleIndex } from '../../../../../../util/list-access';
import { filterIndices, getAccessOperands, resolveSingleIndex } from '../../../../../../util/containers';
import { getConfig } from '../../../../../../config';

interface TableAssignmentProcessorMarker {
Expand Down Expand Up @@ -139,6 +138,7 @@ function processNumberBasedAccess<OtherInfo>(
}]);

const fnCall = processKnownFunctionCall({ name, args, rootId, data, forceArgs: config.forceArgs });

/* recover the environment */
if(existing !== undefined) {
data.environment.current.memory.set(':=', existing);
Expand All @@ -150,6 +150,11 @@ function processNumberBasedAccess<OtherInfo>(
rootId
);
}

if(getConfig().solver.pointerTracking) {
referenceAccessedIndices(args, data, fnCall, rootId, true);
}

return fnCall;
}

Expand Down Expand Up @@ -192,50 +197,58 @@ function processStringBasedAccess<OtherInfo>(

const fnCall = processKnownFunctionCall({ name, args: newArgs, rootId, data, forceArgs: config.forceArgs });

if(getConfig().solver.pointerTracking) {
referenceAccessedIndices(newArgs, data, fnCall, rootId, false);
}

return fnCall;
}

function referenceAccessedIndices<OtherInfo>(
newArgs: readonly RFunctionArgument<OtherInfo & ParentInformation>[],
data: DataflowProcessorInformation<OtherInfo & ParentInformation>,
fnCall: ProcessKnownFunctionCallResult,
rootId: NodeId,
isIndexBasedAccess: boolean,
) {
// Resolve access on the way up the fold
const nonEmptyArgs = newArgs.filter(arg => arg !== EmptyArgument);
const accessedArg = nonEmptyArgs.find(arg => arg.info.role === RoleInParent.Accessed);
const accessArg = nonEmptyArgs.find(arg => arg.info.role === RoleInParent.IndexAccess);
const { accessedArg, accessArg } = getAccessOperands(newArgs);

if(accessedArg === undefined || accessArg === undefined) {
return fnCall;
return;
}

if(getConfig().solver.pointerTracking) {
let accessedIndicesCollection: ContainerIndicesCollection;
// If the accessedArg is a symbol, it's either a simple access or the base case of a nested access
if(accessedArg.value?.type === RType.Symbol) {
accessedIndicesCollection = resolveSingleIndex(accessedArg, accessArg, data.environment);
} else {
// Higher access call
const underlyingAccessId = accessedArg.value?.info.id ?? -1;
const vertex = fnCall.information.graph.getVertex(underlyingAccessId);
const subIndices = vertex?.indicesCollection
?.flatMap(indices => indices.indices)
?.flatMap(index => (index as ContainerParentIndex)?.subIndices ?? []);
if(subIndices) {
accessedIndicesCollection = filterIndices(subIndices, accessArg);
}
let accessedIndicesCollection: ContainerIndicesCollection;
// If the accessedArg is a symbol, it's either a simple access or the base case of a nested access
if(accessedArg.value?.type === RType.Symbol) {
accessedIndicesCollection = resolveSingleIndex(accessedArg, accessArg, data.environment, isIndexBasedAccess);
} else {
// Higher access call
const underlyingAccessId = accessedArg.value?.info.id ?? -1;
const vertex = fnCall.information.graph.getVertex(underlyingAccessId);
const subIndices = vertex?.indicesCollection
?.flatMap(indices => indices.indices)
?.flatMap(index => (index as ContainerParentIndex)?.subIndices ?? []);
if(subIndices) {
accessedIndicesCollection = filterIndices(subIndices, accessArg, isIndexBasedAccess);
}
}

// Add indices to vertex afterward
if(accessedIndicesCollection) {
const vertex = fnCall.information.graph.getVertex(rootId);
if(vertex) {
vertex.indicesCollection = accessedIndicesCollection;
}
// Add indices to vertex afterward
if(accessedIndicesCollection) {
const vertex = fnCall.information.graph.getVertex(rootId);
if(vertex) {
vertex.indicesCollection = accessedIndicesCollection;
}

// When access has no access as parent, it's the top most
const rootNode = data.completeAst.idMap.get(rootId);
const parentNode = data.completeAst.idMap.get(rootNode?.info.parent ?? -1);
if(parentNode?.type !== RType.Access) {
// Only reference indices in top most access
referenceIndices(accessedIndicesCollection, fnCall, name.info.id);
}
// When access has no access as parent, it's the top most
const rootNode = data.completeAst.idMap.get(rootId);
const parentNode = data.completeAst.idMap.get(rootNode?.info.parent ?? -1);
if(parentNode?.type !== RType.Access) {
// Only reference indices in top most access
referenceIndices(accessedIndicesCollection, fnCall, rootId);
}
}

return fnCall;
}

/**
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ import type { ForceArguments } from '../common';
import type { REnvironmentInformation } from '../../../../../environments/environment';
import type { DataflowGraph } from '../../../../../graph/graph';
import { getAliases } from '../../../../../environments/resolve-by-name';
import { addSubIndicesToLeafIndices } from '../../../../../../util/list-access';
import { addSubIndicesToLeafIndices, resolveIndicesByName } from '../../../../../../util/containers';
import { getConfig } from '../../../../../../config';

function toReplacementSymbol<OtherInfo>(target: RNodeWithParent<OtherInfo & ParentInformation> & Base<OtherInfo> & Location, prefix: string, superAssignment: boolean): RSymbol<OtherInfo & ParentInformation> {
Expand Down Expand Up @@ -270,6 +270,15 @@ export function markAsAssignment(
// support for tracking indices
// Indices were defined for the vertex e.g. a <- list(c = 1) or a$b <- list(c = 1)
indicesCollection = information.graph.getVertex(sourceIds[0])?.indicesCollection;

// support assignment of container e.g. container1 <- container2
// defined indices are passed
if(!indicesCollection) {
const node = information.graph.idMap?.get(sourceIds[0]);
if(node && node.type === RType.Symbol) {
indicesCollection = resolveIndicesByName(node.lexeme, information.environment);
}
}
}
// Indices defined by replacement operation e.g. $<-
if(config?.indicesCollection !== undefined) {
Expand All @@ -293,14 +302,12 @@ export function markAsAssignment(
}
}
information.graph.addEdge(nodeToDefine, rootIdOfAssignment, EdgeType.DefinedBy);
if(getConfig().solver.pointerTracking) {
// kinda dirty, but we have to remove existing read edges for the symbol, added by the child
const out = information.graph.outgoingEdges(nodeToDefine.nodeId);
for(const [id, edge] of (out ?? [])) {
edge.types &= ~EdgeType.Reads;
if(edge.types == 0) {
out?.delete(id);
}
// kinda dirty, but we have to remove existing read edges for the symbol, added by the child
const out = information.graph.outgoingEdges(nodeToDefine.nodeId);
for(const [id, edge] of (out ?? [])) {
edge.types &= ~EdgeType.Reads;
if(edge.types == 0) {
out?.delete(id);
}
}
}
Expand Down
Loading
Loading