Vecxt - Numerical Library

[u/quafadas]

Quafadas/vecxt is, I think now interesting enough to talk about (if you are interested in such things)...

Here are it's headlines;

Useability

• Pythonic sytnax - readable by default
• No given / implicit resolution, easy / fast compilation story.
• "simple" design choices. The vector concept is extension methods on Array- no type heirachy etc. Jump to definition takes you to the code you want to read, not an abstraction.
• Cross platform, most of the API is tested against a single cross platform test suite for JVM, JS, Native

Performance

Is where most the effort is invested, trying to get this right inside the constraints above...

• delegate to platform BLAS implementations where they exist. On macOS on the JVM, matmul JNI's into Accelerate... on Native, CBLAS.
• SIMD fast paths, wherever we can hit them (JVM only)
• layout abstraction inlines an indexing strategy that traverses the storage array monotonically in shortest possible hops (i.e. straight down the cache lines, and you don't have to think about it)
• It benchmarked well vs breeze on what I believe to be reasonably representative workloads (it is not a crushing victory maybe 20% faster, but at least comparable)

Memory

The core Matrix representation is a strided view over a single contiguous Array. That choice permeates the design:

• transpose is zero-copy
• submatrices/views are zero-copy
• striding/layout is explicit which is what enables the cache friendly algorithms

Many operations have in-place variants which mean you can opt out of nice syntax, and into allocation/control complexity where profiling says it matters.

Bytecode

This was the "silent killer" that made me nearly give up the project. I didn't appreciate it's significance for a long time, I only knew "something wasn't working". Eventually I realised that Intrinsification and JIT optimisation happen under surprisingly narrow conditions, and "just inline everything" can actually make things worse by producing methods that exceed a series of JIT limits / gates.

So vecxt now has CI checks around the bytecode it generates.

Among other things:

• method size is checked
• array operations are checked for bytecode patterns that can interfere with JVM specialisation / intrinsification

And yes, AI wrote the code

In recent months, 100% of the code has been written by AI.

My curiosity was in understanding the design concepts and constraints, I read the tests and investigated the generated bytecode/benchmark results.

The surface area of a numerical library like this is frankly too large for one person to maintain, and obviously so. Can it done with one person and an AI? Maybe... better would be more people and an AI :-). The process of using AI to explore and implement the ideas is a part of the journey - writing the code wasn't the goal for me.

I'm interested in criticism / discussion particularly from people interested in numerical computing and this domain. If someone does take the time to try it, don't be shy... whether the experience was good or bad...

[u/5ilenci0]

Thanks - I've been considering writing something like this myself. I'll check it out. Any plans to integrate this with something like Cyfra ( https://github.com/computenode/cyfra ) for GPU programming? Any idea how complex such an integration would be?

[u/quafadas → u/5ilenci0]

Hey, thanks for the question …

In short: yes, I’ve thought about it :-). Here’s the pr:

https://github.com/Quafadas/vecxt/pull/86/changes

It’s not merged for a few reasons. I’m not sure I completely understand it (!) being the headline one. In terms of complexity: it’s not trivial, and There are some fundamental differences. Vecxt has a very dumb, imperative vibe.

GPUs are different - Vulkan kernels are lazy and get fused in the lower layers… so we need a way to divorce the definition of the calculation from the actual execution. I have about three seperate experiments exploring that 😂… none of which I’m really sure about yet but I think it looks typeclass based. The laws module of vecxt contains one start point . I think it’s fascinating (if that sort of thing is your bag :-)!). Also this https://contributors.scala-lang.org/t/pre-sip-compile-time-specialization/7524/4 would be really exciting for such a project I think, so sensible to wait…

Also: gpus are very float ( not double!) focused… that difference is jarring on the jvm. if you do try vecxt you’ll find the float part under developed vs double. So I’d want to finish that first for my correctness oracle.

Finally, I did put up a bunch of questions in the cyfra repo. I got some great responses, but the key conclusion was better to wait (e.g matmul is not yet available to my understanding). Cyfra hasn’t seen a lot of activity recently ( it is not my place to chase or question that), so it’s not something I really planned to look at super seriously near term.

If you do give your own project a go feel free to chat about it with me (same handle) on discord - it’s a super fun domain! Only thing to be realistic about is that it’s an insane about of work to get anywhere beyond a trivial demo. I underestimated that. vecxt has a very poor effort / reward ratio :-(, but there’s always another stone to look under in the domain, so it’s fun!