Graphing using data in a separate file from a local path (rust executable)

[yuki-tsubaki]

Context

v5 schema
vl-convert 1.6.1
On GNU/Linux

Issue/Question

I am trying to plot something with vl-connect
When I embed the data directly in the spec everything works properly and I get a nice svg showing the plot.

However since I often show multiple visualizations that use the same data (and just to make things cleaner) I would prefer to keep the data in a different file and reference it with the url.

I.e. {"data":{"url":"path/to/data.json"}...}

However, when I do this I never can get it to actually access the data and I end up with an empty scatter plot.

I've tried various permutations with the data in different places, full paths, relative paths to different directories, and more but I still haven't been able to get anything to work.

Can someone help and point out what I am missing here?
Thanks in advance.

[jonmmease]

Hi @yuki-tsubaki, thanks for the idea. I'll need to play with it some, but have you tried using file:// urls? I think Vega defaults to treating paths without a protocol as being http.

We may be able to do something by customizing the Vega loader here:

vl-convert/vl-convert-rs/src/converter.rs

Lines 352 to 368 in 9e72c4e

	const loader = vega.loader({ mode: 'http', baseURL });
	const originalHttp = loader.http.bind(loader);
	if (allowedBaseUrls != null) {
	loader.http = async (uri, options) => {
	const parsedUri = new URL(uri);
	if (
	allowedBaseUrls.every(
	(allowedUrl) => !parsedUri.href.startsWith(allowedUrl),
	)
	) {
	errors.push(`External data url not allowed: ${uri}`);
	throw new Error(`External data url not allowed: ${uri}`);
	}
	return originalHttp(uri, options);
	};
	}

And there's some info the Vega loader at https://github.com/vega/vega/tree/main/packages/vega-loader.

[yuki-tsubaki]

Thanks for such a fast reply.
I have tried multiple different notations for the path to the data file.

I have tried with and without the file:// protocol for absolute (from root--e.g. /home/...data/hp-mpg-plot-data.json), relative (to the working directory--e.g. data/hp-mpg-plot-data.json), and relative (to the spec file--e.g. ../data/hp-mpg-plot-data.json) paths.

As a side note, (I have no clue on how actually practical this would be for the architecture this is using), having error messages would be nice. I see that the underlying JS does log failed loading when I put in a fake, non-data, or unavailable url, so if there were a way to send these to the standard output (when using a --debug flag for instance) would be helpful in figuring this stuff out.

[mgilbir]

@jonmmease I've been toying with something like that in this PR, but I didn't have time to finish it.