I must preface this by saying how impressed I am by the current state of the art in AI, and neural networks in general. However. I find them to be computationally, and financially obscene. Modern programmers have been raised in an era of cheap processing power, RAM, storage, communication speed – you name it, the hardware is many orders of magnitude better than what I had for my original education. They have no sense for efficiency, no true thermodynamic understanding of the cost of bloat and wasted calculations. This can be improved. <\/p>\n\n\n\n
Some design principles should be named in advance.<\/p>\n\n\n\n
The current NN models consist of planes of computational units. Between two adjacent planes, everything connects to everything. Every calculation engages every single elemental unit, every time. This is both unlike biologic neurons and completely unnecessary for useful work. Obviously, trying to directly model axons and dendrites, glial tissue, ion channels and other organic signaling paths might make for an interesting exercise, but will not solve the computational issues that need to be addressed.<\/p>\n\n\n\n
The goals are:<\/p>\n\n\n\n
To mimic what is known to work, while respecting the digital substrate, and the necessary map of translation between the two domains.<\/p>\n\n\n\n
To use a principle of elegance, reduce everything unnecessary until the only parts that remain are fundamentally required. <\/p>\n\n\n\n
No reinventing the wheel. Repurpose known models in new combinations, new coordinate systems.<\/p>\n\n\n\n
So, with those named, I would like to propose the following:<\/p>\n\n\n\n
Use phase space as a defining aspect, every unit must exist within a complex coordinate system. This immediately simplifies the math for things like frequency, phase, dimensionality, and connectivity rules. x + iy, and then extended into quaternions. Dimensionality should be a trivial parameter adjustment in the coordination engine.<\/p>\n\n\n\n
The interactions between units should resemble music, alignment of phase, alignment of resonance.<\/p>\n\n\n\n
Simple lookup tables beats brute force computation every time.<\/p>\n\n\n\n
The rules should allow for divergence of function across a vast array of vexels, smaller subgroups must have rotational shifts in the complex space to prevent unwanted crosstalk. The capacity for crosstalk can also be defined as nearly resonant groupings. Reduced amplitude, but not complete orthogonality.<\/p>\n\n\n\n
The rule for activation is local, but defined by close phase relations rather than simple summing.<\/p>\n\n\n\n
Single bit state flags and many hysteresis clocks should be inherent in the design of each unit. Modifiable by history and frequency of activation. Op-amp circuits can be used as abstract guidelines for some of this.<\/p>\n\n\n\n
<\/p>\n","protected":false},"excerpt":{"rendered":"
I must preface this by saying how impressed I am by the current state of the art in AI, and neural networks in general. However. I find them to be computationally, and financially obscene. Modern programmers have been raised in an era of cheap processing power, RAM, storage, communication speed – you name it, the […]<\/p>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-20","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/vexel.codes\/index.php?rest_route=\/wp\/v2\/pages\/20","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/vexel.codes\/index.php?rest_route=\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/vexel.codes\/index.php?rest_route=\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/vexel.codes\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/vexel.codes\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=20"}],"version-history":[{"count":9,"href":"https:\/\/vexel.codes\/index.php?rest_route=\/wp\/v2\/pages\/20\/revisions"}],"predecessor-version":[{"id":45,"href":"https:\/\/vexel.codes\/index.php?rest_route=\/wp\/v2\/pages\/20\/revisions\/45"}],"wp:attachment":[{"href":"https:\/\/vexel.codes\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=20"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}