Children at the 38th Annual Nordheim Easter Egg Hunt met Penny Cottontail
To access content, please login or purchase a subscription
Read Cuero Record
Read Yorktown News-View
© 2025 DeWitt County Today
Metrics details
a Illustration of the distribution of solutions generated by optimal and non-optimal Ising machines for different COP complexity: Low (L)
An ideal Ising/QUBO solver produces distribution of solutions that is concentrated near the SOTA and has the potential to produce novel
previously unknown solution that is closer to the Ising ground state; A MAX-CUT problem defined over a b graph with weights Qij which is decomposed into c pairs of ON–OFF neurons by NeuroSA
d Each ON–OFF integrate-and-fire neurons are coupled to each other by an excitatory synapse with weight A and the pair is connected differentially to other ON–OFF neuron pairs through the synaptic weights Qij, −Qij
The thresholds for both ON–OFF neurons are dynamically adjusted by an e FN annealer which comprises an FN integrator
an exponentially distributed noise source \({{{\mathcal{N}}}}_{{\rm {n}}}^{{\rm {E}}}\) and a Bernoulli noise source \({{{\mathcal{N}}}}_{{\rm {n}}}^{\rm {{B}}}\); Illustration of NeuroSA dynamics for a MAX-CUT graph with 10 vertices connected by a weight matrix Q shown in (f)
g Evolution of the distance between the solutions generated by NeuroSA to the two known ground state solutions at a given time-instant which highlights the escape mechanisms in the high- and low-temperature regimes
h Raster plot of aggregated spiking activity generated by the ON and OFF neuron pairs
and i visualization of the NeuroSA trapping and escape dynamics using a principal component analysis (PCA)-based projection of the network spiking activity estimated within a moving time-window
How can optimal simulated annealing algorithms be mapped onto large-scale neuromorphic architectures
The key underpinnings of any neuromorphic architecture are: (a) asynchronous (or Poisson) dynamics that are generated by a network of spiking neurons; and (b) efficient and parallel routing of spikes/events between neurons across large networks
Both these features are essential for solving the Ising problem and efficient mapping of SA onto neuromorphic architecture
In its general form Ising problem minimizes a function (or a Hamiltonian) H(s) of the spin state vector s according to
is associated with one of the D vertices in the graph \({{\mathcal{G}}}\)
The graph’s edges are represented by a matrix \({{\bf{Q}}}\in {{\mathbb{R}}}^{D\times D}\)
wherein Qij signifies the weight associated with the edge connecting vertices i and j
the objective of the MAX-CUT problem is to partition the vertices into two classes
maximizing the number of edges between them
If an ideal asynchronous operation is assumed (see “Methods” subsection “ Asynchronous ising machine model”)
only one spin (say the pth spin) changes its state by Δsp,n ∈ { −1
we chose the digital emulation because of the precision required for SA in the low-temperature regime
The FN dynamics can then be combined with the independent identically distributed (i.i.d.) random variables \({{{\mathcal{N}}}}_{p,n}^{{\rm {E}}}\) and \({{{\mathcal{N}}}}_{p,n}^{{\rm {B}}}\) to determine the dynamic firing threshold μn,p for each ON–OFF integrate-and-fire neuron pair p
\({{{\mathcal{N}}}}_{p,n}^{{\rm {E}}}\) is drawn from an exponential distribution whereas \({{{\mathcal{N}}}}_{p,n}^{{\rm {B}}}\) is drawn from a Bernoulli distribution with values {0, 1}
The choice of the two distributions ensures that every neuron has a finite probability of firing
which is equivalent to satisfying the irreducibility and aperiodicity conditions in SA
and this is achieved without significant tuning of hyperparameters
a Convergence plot showing steady increase in the solution quality with the inset showing fluctuations near 3050 cuts which is the current SOTA for this graph
b Dynamics of the firing threshold with inset showing sparse but large fluctuations that trigger escape mechanisms
c Plot showing the number of active neurons decaying following \(\sim \frac{1}{\log t}\) without the contribution of the Bernoulli r.v
d PCA trajectory of the NeuroSA dynamics where the initial (high temperature) regime follows a path defined by the network gradient and the trajectory near convergence (or low-temperature path) exhibits expanding exploration of the solution space
e Distribution of the G15 solutions obtained for different annealing schedules (\({{\rm {e}}}^{-t},{(\log t)}^{-1},{t}^{-1}\)) and noise statistics (exponential—denoted by \({{{\mathcal{N}}}}^{{\rm {E}}}\)
Gaussian—denoted by \({{{\mathcal{N}}}}^{{\rm {G}}}\)
and Uniform—denoted by \({{{\mathcal{N}}}}^{{\rm {U}}}\))
Only for an FN annealer and an exponentially distributed noise NE
the distribution of solutions obtained by NeuroSA is more concentrated around the SOTA
Next, the NeuroSA architecture was benchmarked for solving MAX-CUT problems on different Gset graphs. Figure 3 provides a detailed evaluation of the NeuroSA algorithm’s performance on the Gset benchmarks
with results generated using both traditional CPU-based and the SpiNNaker2 platform
The architecture is configured similarly for both hardware platforms and across all benchmark tests
as it demonstrates that NeuroSA’s performance is robust across and agnostic to different MAX-CUT graph complexities
it obviates painstaking hyperparameter tuning for each set of graphs or problems
the parallel search results in a more concentrated distribution around the SOTA solution than the long-running singleton approach
the parallel NeuroSA instances take different trajectories to the solution and the aggregation of multiple searches results in amplifying the probability of reaching the neighborhood of the SOTA/ground state solution within a finite time constraint
we would like to point out that the optimal annealing schedule promises global optimality only in the asymptotic (exponential) time limit
improvements on the current solution to reach the ground state using a parallel approach would still rely on long run-time of one of many singleton NeuroSA instances
to 1011 iterations to see if NeuroSA could find a novel solution for a few sets of MAX-CUT problems
within finite run-time (both on CPU and SpiNNaker2)
NeuroSA was unable to discover solutions that exceeded the SOTA for the Gset/MAX-CUT benchmarks
As it becomes harder to find better solutions
the ratio tends to unity (run-time becomes exponential or sub-exponential) This is shown by the extrapolation curve and the transition point A
which might highlight the point of diminishing returns
This could be taken as a hardware-agnostic stopping criterion (polynomial run-time) for a COP
the SpiNNaker2 implementation outperforms the CPU-based implementation in terms of energy-to-solution for the same workload
this performance advantage is evident even if the current SpiNNaker2 implementation of NeuroSA is not fully optimized
Solutions are ordered according to complexity metrics: a number of graph vertices
M = {100, 250, 500} and H = {1000, 2500, 5000}
we proposed a neuromorphic architecture called NeuroSA that is functionally isomorphic to a simulated annealing optimization engine
The isomorphism allows mapping optimal SA algorithms to neuromorphic architectures
providing theoretical guarantees of asymptotic convergence to the Ising ground state
The core computational element of NeuroSA is formed by an ON–OFF integrate-and-fire neuron pair that can be implemented on any standard neuromorphic hardware
NeuroSA can exploit the computational power of both existing and upcoming large-scale neuromorphic platforms
Inside each ON–OFF neuron pair is an annealer whose stochastic properties are dictated by a Fowler–Nordheim (FN) dynamical system
the neuron model and the FN annealer generate population activity that emulates the sequential acceptance and rejection dynamics of the SA algorithm
Any choice of distribution other than the exponential distribution for the r.v
\({{{\mathcal{N}}}}_{n}^{{\rm {E}}}\) will violate the SA’s detailed balance criterion and hence the network might not encode a steady-state distribution
the energy landscape of the resulting H(s) is more complex and hence would require different choices of T0 and the simulation time to achieve SOTA solutions
This region of convergence corresponds to the low-temperature regime where it is important to explore distant states and at the same time accept proposals (or produce spikes) only when the network energy decreases
the time needed to achieve a unit gain in the quality of the solution increases with time
with the last gain consuming the majority of the entire simulation duration
accelerating NeuroSA’s initial convergence using a low-temperature start might not significantly reduce the overall time-to-solution when the goal is to approach the asymptotic ground state
the approach does enhance the efficiency of the NeuroSA to approach SOTA solutions under real-time constraints
but only in conjunction with analog-to-digital converters with more than 16 bit precision
The scope of this work is to present the algorithmic advancement in developing an asynchronous neuromorphic architecture that can utilize the FN-annealing dynamics
the annealing schedule should follow a schedule that is slower than c/log(t) (or FN tunneling) dynamics to inherit the asymptotic global optimization convergence properties
the scaling parameter c needs to be selected to achieve the best trade-off between faster convergence and the quality of the solution for a given optimization problem
a schedule that follows \(c/{\log }^{1/2}(t)\) or \(c/\log \log (t)\) dynamics would also guarantee asymptotic convergence to the ground state
is one-shot solver unlike AI/ML inference engines
where the overhead of physical instantiation can be amortized over repetitive runs
Since one of the focus of NeuroSA is to explore new solutions as it asymptotically converges to the ground state
it is expected that NeuroSA implementations would require long run-times
As new/better solutions are produced by the solver they can be read out and used in other applications
QUBO and Ising formulations are interchangeable through a variable transform \(s\leftrightarrow \frac{1+s}{2}\)
we consider the following optimization problem
where \({{\bf{s}}}=\left[{s}_{1},{s}_{2},.. ,{s}_{D}\right]\) denotes a spin vector comprising of binary optimization variables. Because \({s}_{j}^{2}=1,\forall j=1...D\), Eq. (3) is equivalent to
Note the matrix Q can be symmetrized by \({{\bf{Q}}}\leftarrow \frac{1}{2}({{\bf{Q}}}+{{{\bf{Q}}}}^{\intercal })\) without changing the solution to Eq. (3)
Let the vector s at time instant n be denoted by sn and the change in s be denoted as Δsn
where Δsn = {−1, 0, +1}D and Δsj,n sj,n−1 = −1
∀ j = 1. . . D ensures that the spin either flips or remains unchanged
where the set \({{\mathcal{C}}}=\{i:\Delta {s}_{i,n}\, \ne \, 0\}\) denotes the neurons that do not fire at time-instant n. Solving Eq. (3) involves solving the sequentially sub-problem: ∀ n
+1}D such that \({\sum }_{p\in {{\mathcal{C}}}}\Delta {s}_{p,n}[{\sum }_{j\notin {{\mathcal{C}}}}{Q}_{pj}{s}_{j,n}]\
which in itself is a combinatorial problem
the problem of searching for the set of firing neurons can be simplified
only one of the neurons can emit a spike at any time instant n (due to Poisson statistics)
where un is a uniformly distributed r.v. between \(\left[0,1\right]\), and B > 1 is a hyper-parameter, Tn > 0 denotes the temperature at time-instant n. Eq. (11) is equivalent to
which corresponds to the spiking criterion for an ON neuron and
which corresponds to the spiking criterion for an OFF neuron. Introducing a RESET parameter \(A\gg | {T}_{n}\log \left(\frac{{u}_{n}}{B}+\epsilon \right)|\), Eqs. (15) and (16) are equivalent to the ON neuron model
The variables \({v}_{p,n}^{+},{v}_{p,n}^{-}\) represent the membrane potentials of the ON–OFF integrate-and-fire neurons at the time instant n
To ensure that all neurons are equally likely to be selected (to satisfy the ergodicity property of SA)
where \({\mu }_{n,p}={T}_{n}{{{\mathcal{N}}}}_{n}^{{\rm {E}}}+A{{{\mathcal{N}}}}_{p,n}^{{\rm {B}}}\) denotes the shared noisy threshold between the pth pair of ON–OFF neurons at time instance n
The ON–OFF construction ensures that \(\Delta {s}_{p,n}^{+}\Delta {s}_{p,n}^{-}=0,\forall p,n\)
which leads to the following fundamental ON–OFF integrate-and-fire neuron model of NeuroSA which is summarized as: The ON–OFF neuron’s membrane potentials \({v}_{p,n}^{+},{v}_{p,n}^{-}\in {\mathbb{R}}\) evolve as
where A > 0 is a constant that represents an excitatory synaptic coupling between the ON and the OFF neurons, as shown in Fig. 1d
The ON and OFF neurons generate a spike when their respective membrane potential exceeds a time-varying noisy threshold μp,n according to
after which the membrane potentials are RESET by subtraction according to
In ref. 56 it was shown that a temperature annealing schedule of the form
where C is a normalizing constant. Differentiating Eq. (29) one obtains the dynamical systems model
The sampling period in the discrete-time model was chosen by adjusting C and to fit the optimal scheduling requirements of the NeuroSA algorithm
The mean of the random variable \({{{\mathcal{N}}}}_{p,n}^{{\rm {E}}}\) was chosen to be −0.916
and Section S5 for SpiNNaker2 implementation
The results in each run were normalized with respect to the SOTA and then grouped as a histogram
The probability density function (pdf) was generated using ksdensity
The COP solutions, the algorithm execution time, the power and the energy data for the CPU and SpiNNaker2 imeplementation of NeuroSA generated in this study are available at https://github.com/aimlab-wustl/neuroSA
The specific MATLAB and Python codes used in simulation/emulation studies on the CPU platform are available at https://github.com/aimlab-wustl/neuroSA
Since the SpiNNaker2 stack is still under development
the software implementation on SpiNNaker2 will not be made available publicly but will be available upon requests
On the computational complexity of ising spin glass models
Lucas, A. Ising formulations of many np problems. Front. Phys. 2, https://doi.org/10.3389/fphy.2014.00005 (2014)
Ising machines as hardware solvers of combinatorial optimization problems
Experimental investigation of performance differences between coherent ising machines and a quantum annealer
Application of ising machines and a software development for ising machines
Coherent quantum annealing in a programmable 2000 qubit ising chain
Optoelectronic coherent ising machine for combinatorial optimization problems
An enhanced 1440 coupled CMOS oscillator network to solve combinatorial optimization problems
In 2023 IEEE 36th International System-on-Chip Conference (SOCC) (ed
A CMOS-compatible oscillation-based VO2 Ising machine solver
Power-efficient combinatorial optimization using intrinsic noise in memristor hopfield neural networks
Jiang, M., Shan, K., He, C. & Li, C. Efficient combinatorial optimization by quantum-inspired parallel annealing in analogue memristor crossbar. Nat. Commun. 14, https://doi.org/10.1038/s41467-023-41647-2 (2023)
Fahimi, Z., Mahmoodi, M., Nili, H., Polishchuk, V. & Strukov, D. Combinatorial optimization by weight annealing in memristive Hopfield networks. Sci. Rep. 11, https://doi.org/10.1038/s41598-020-78944-5 (2021)
Optimised simulated annealing for ising spin glasses
A new computing architecture using ising spin model implemented on fpga for solving combinatorial optimization problems
In 2017 IEEE 17th International Conference on Nanotechnology (IEEE-NANO) 256–258 (IEEE
A 20k-spin ising chip to solve combinatorial optimization problems with cmos annealing
Computing architecture to perform approximated simulated annealing for ising models
In 2016 IEEE International Conference on Rebooting Computing (ICRC) 1–8 (IEEE
Quantum annealing in the transverse ising model
Colloquium: quantum annealing and analog quantum computation
A quantum adiabatic evolution algorithm applied to random instances of an np-complete problem
Combinatorial optimization by simulating adiabatic bifurcations in nonlinear hamiltonian systems
Evaluating spintronics-compatible implementations of ising machines
Singh, N.S. et al. Cmos plus stochastic nanomagnets enabling heterogeneous computers for probabilistic inference and learning. Nat. Commun. 15, https://doi.org/10.1038/s41467-024-46645-6 (2024)
Energy efficiency of machine learning in embedded systems using neuromorphic hardware
Neuromorphic artificial intelligence systems
Hierarchical address event routing for reconfigurable large-scale neuromorphic systems
Loihi: a neuromorphic manycore processor with on-chip learning
Spinnaker 2: a 10 million core processor system for brain simulation and machine learning
Performance comparison of the digital neuromorphic hardware spinnaker and the neural network simulation software nest for a full-scale cortical microcircuit model
Real-time cortical simulation on neuromorphic hardware
Memory-efficient deep learning on a spinnaker 2 prototype
Programming spiking neural networks on intel’s loihi
SpiNNaker2: a large-scale neuromorphic system for event-based and asynchronous machine learning
Machine Learning with New Compute Paradigms Workshop at NeurIPS (MLNPCP) (2023)
Energy estimates across layers of computing: from devices to large-scale applications in machine learning for natural language processing
In 2023 IEEE High Performance Extreme Computing Conference (HPEC) 1–6 (IEEE
Neural networks and physical systems with emergent collective computational abilities
The free-energy principle: a unified brain theory
The benefits of noise in neural systems: bridging theory and experiment
Ion channel stochasticity may be critical in determining the reliability and precision of spike timing
Local dendritic activity sets release probability at hippocampal synapses
Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communication
Oscillatory neurocomputers with dynamic connectivity
An introduction to restricted Boltzmann machines
Mehta, D., Rahman, M., Aono, K. & Chakrabartty, S. An adaptive synaptic array using fowler-nordheim dynamic analog memory. Nat. Commun. 13, https://doi.org/10.1038/s41467-022-29320-6 (2022)
Neuromorphic scaling advantages for energy-efficient random walk computations
Optimal algorithms and inapproximability results for every CSP
40th Annual ACM Symposium on Theory of Computing
STOC’08 245–254 (Association for Computing Machinery
On the unique games conjecture (invited survey)
In 2010 IEEE 25th Annual Conference on Computational Complexity 99–121 (IEEE
Goemans–Williamson maxcut approximation algorithm on loihi
B.) 1–5 (Association for Computing Machinery
Stochastic neuromorphic circuits for solving MAXCUT
In 2023 IEEE International Parallel and Distributed Processing Symposium (IPDPS) (ed
Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming
Electron emission in intense electric fields
Self-powered timekeeping and synchronization using fowler-nordheim tunneling-based floating-gate integrators
The neurobench framework for benchmarking neuromorphic computing algorithms and systems
Intensity versus identity coding in an olfactory system
Ye, Y. The Gset Dataset https://web.stanford.edu/yyye/yyye/Gset/ (2003)
Transitivity in structural models of small groups
Expectation values from the single-layer quantum approximate optimization algorithm on ising problems
Epos lhc: test of collective hadronization with data measured at the cern large hadron collider
Mehta, D., C.S., Aono, A. A self-powered analog sensor-data-logging device based on Fowler–Nordheim dynamical systems. Nat. Commun. 11, https://doi.org/10.1038/s41467-020-19292-w (2020)
Rueckauer, B., Lungu, I.-A., Hu, Y., Pfeiffer, M. & Liu, S.-C. Conversion of continuous-valued deep networks to efficient event-driven networks for image classification. Front. Neurosci. 11, https://doi.org/10.3389/fnins.2017.00682 (2017)
Dynamic optimization of odor representations by slow temporal patterning of mitral cell activity
A sparsity-driven backpropagation-less learning framework using populations of spiking growth transform neurons
Download references
This work is supported in part by research grants from the US National Science Foundation: ECCS:2332166 and FET:2208770
would like to acknowledge the financial support by the Federal Ministry of Education and Research of Germany in the program of “Souverän
acknowledge the EIC Transition under the ”SpiNNode” project (grant number 101112987)
also acknowledge the Horizon Europe project “PRIMI" (Grant number 101120727)
acknowledges a contribution from the Italian National Recovery and Resilience Plan (NRRP)
funded by the European Union-NextGenerationEU (Project IR0000011
acknowledges Department of Energy’s Office of Science contract DE-AC02-76SF00515 with SLAC through an Annual Operating Plan agreement WBS 2.1.0.86 from the Office of Energy Efficiency and Renewable Energy's Advanced Manufacturing and Materials Technology Office and the institutional support from SLAC National Laboratory
Department of Electrical and Systems Engineering
Subhankar Bose & Shantanu Chakrabartty
Department of Electrical and Electronic Engineering
International Centre for Neuromorphic Engineering
Redwood Center for Theoretical Neuroscience and Helen Wills Neuroscience Institute
Chair of Highly-Parallel VLSI-Systems and Neuro-Microelectronics
Department of Electrical and Computer Engineering
Department of Electrical and computer engineering
Scads.AI: Center for Scalable Data Analytics and Artificial Intelligence
participated in a workgroup titled Quantum-inspired Neuromorphic Systems at the Telluride Neuromorphic and Cognitive Engineering (TNCE) workshop in 2023
and the outcomes from the workgroup have served as the motivation for this work
formulated the asynchronous ON–OFF neuron model with the FN-annealer; Z.C
benchmarked NeuroSA on different MAX-CUT graphs; Z.X
implemented the first version of SA algorithm; S.C
proposed the use of spike events to reduce communication bottleneck in NeuroSA; M.A.
optimized SpiNNaker2 for MAX-CUT benchmarks; All authors/co-authors contributed to proof-reading and writing of the manuscript
SpiNNaker2 is a neuromorphic hardware accelerator platform by SpiNNcloud Systems
and international patents associated with FN-based dynamical systems
and the rights to the intellectual property are managed by Washington University in St
The remaining authors declare no competing interests
reviewer(s) for their contribution to the peer review of this work
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations
Download citation
DOI: https://doi.org/10.1038/s41467-025-58231-5
Anyone you share the following link with will be able to read this content:
a shareable link is not currently available for this article
Sign up for the Nature Briefing: AI and Robotics newsletter — what matters in AI and robotics research
« Back
If you are the site owner (or you manage this site), please whitelist your IP or if you think this block is an error please open a support ticket and make sure to include the block details (displayed in the box below)
so we can assist you in troubleshooting the issue
I had always wanted to come to England [says Thomas von Nordheim
I was a creative child and I hoped to become an architect
but at some point realised that creating in two dimensions did not satisfy my aspirations
I watched a haute couture fashion show in Düsseldorf aged 17 and decided I wanted to work in this world
There were a few really cool clubs in Düsseldorf in the 1980s
We had the extravagant fashions of Jean Paul Gaultier and so on
Junior Vanessa Lopez tips the ball back over
To access content, please login or purchase a subscription
Senior Chapter Conducting competitors include Cody Jones
The Greenland chapter conducting includes Addison Rogers
The Greenland Quiz team includes Addison Rogers
To access content, please login or purchase a subscription
To access content, please login or purchase a subscription
Greystar Real Estate Partners, LLC has secured $183 million in tax-exempt bond financing and completed the acquisitions of Nordheim Court and Radford Court
two on-campus communities at the University of Washington
Nordheim Court and Radford Court are now owned by Provident Resources Group Inc.
Greystar’s Student Living team manages and operates them
Greystar’s Pacific Northwest Construction team will enhance both residential units and common areas over a three-year period
Greystar oversees more than 30,000 residential units in the Seattle area
which also includes two additional off-campus communities catering to UW students
The acquisitions are Phase I of UW’s multiphase plan known as “UH4” to increase housing options and improve housing quality for UW students
UH4 Phase II involves the redevelopment of two additional communities
fully integrated real estate company offering expertise in investment management
and management of rental housing properties globally
Greystar manages and operates an estimated $150 billion of real estate in over 185 markets globally including offices throughout the United States
Greystar is the largest operator of apartments in the United States
and has a robust institutional investment management platform with approximately $35.8 billion of assets under management
including nearly $13.9 billion of assets under development
Greystar was founded by Bob Faith in 1993 with the intent to become a provider of world-class service in the rental residential real estate business
the final demonstration of the MaiSHU project (Multimodal perception and human-machine interfaces of semi-autonomous intelligent systems for humanitarian aid in unsafe and unstructured environments)
took place at the German Federal Armed Forces training area in Nordheim am Main
The aim of the project is to develop innovative technologies for the assisted teleoperation of amphibious SHERP vehicles for humanitarian aid and the delivery of goods in difficult
the World Food Programme (WFP) used the SHERP to deliver food to a village in South Sudan that had been cut off by persistent rain and flooding
The SHERP navigated the dangerous and inaccessible route without a driver
controlled by the Local Mission Operation Centre (LMOC)
which worked closely with the Global Mission Operation Centre (GMOC)
the Bavarian Red Cross (BRK) coordinated the operation of SHERP to evacuate people from a flooded and landslide-affected area
GMOC/ZKI provided the BRK with several situation maps as well as the GMOC web application with crisis information for a comprehensive overview of the situation so that a safe evacuation could be ensured through the use of the SHERP
the AHEAD (Autonomous Humanitarian Emergency Aid Devices) project presented advanced technologies for humanitarian aid and disaster management
Please select what you would like included for printing:
Copy the text below and then paste that into your favorite email application
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply
Service map data © OpenStreetMap contributors
Metrics details
In this paper we present an adaptive synaptic array that can be used to improve the energy-efficiency of training machine learning (ML) systems
The synaptic array comprises of an ensemble of analog memory elements
each of which is a micro-scale dynamical system in its own right
storing information in its temporal state trajectory
The state trajectories are then modulated by a system level learning algorithm such that the ensemble trajectory is guided towards the optimal solution
We show that the extrinsic energy required for state trajectory modulation can be matched to the dynamics of neural network learning which leads to a significant reduction in energy-dissipated for memory updates during ML training
the proposed synapse array could have significant implications in addressing the energy-efficiency imbalance between the training and the inference phases observed in artificial intelligence (AI) systems
a Conventional non-volatile analog memory where transition between analog static states (Wn−1
Wn) dissipates energy (ΔEn); b dynamic analog memory where an external energy (ΔE1
ΔE3) per memory update at time instance t1–t3 and memory retention rate
the synapse’s retention rate should increase as the training progresses such that at convergence or in the inference phase the weights are stored as a non-volatile memory
we show measurement results that verify that the effect of read disturbance is random and the magnitude of the disturbance is less than the precision of the memory update and read-out circuits
a WS (solid line) and WR (dashed line) response under three different operating regimes (zoomed insets: a1
a3) determined by FN-DAM initialization voltage
b–d FN-DAM response (w) calculated as difference between WS and WR voltage values in the three regimes demonstrating different plasticity
Dots are measured datapoints while lines correspond to fit to the data
c Change in WS and WR potentials due to SET and RESET pulses
d DAM response calculated as difference between WS and WR voltages
f FN-DAM response to SET pulses of varying frequency
Error bars indicate standard deviation estimated across 12 devices
The variations could also be reduced by using careful layout techniques and precise timing of the control signals
a DAM response to pulses of different magnitude but same duration (10 ms)
b DAM response to varying number of pulses of 4 V amplitude and 10 ms duration
c Change in DAM response with each pulse of same magnitude (4 V) and duration (10 ms)
d FN-DAM response measured at 100 °C when a SET pulse is applied to 12 different FN-DAM elements (each color corresponds to a different memory element)
a Test data set with randomly initialized decision boundary. b Decision boundary after training. c Evolution of weights (w0 and w1) after 5 epochs. d Input voltage required for initiating a unit change in weights (Ws,0, WR,0, Ws,1, WR,1). e Energy (E(w0), E(w1)) expended in updating the weights (w0 and w1). f Average magnitude of weight update and average energy required for each epoch.
a–c Experimental training on Fisher Iris dataset over five trials: a Five-fold cross-validation accuracy of model over 20 epochs for training set (120 points) and validation set (30 points)
b Total pulses required in implementing weight update for entire synaptic array during each epoch
c Energy per unit capacitance expended in updating the weights
Note: scale of Y-axis is set to match that of panel (b)
Error bars in a–c indicate standard deviation estimated across five trials
e Simulated training on MNIST dataset: d Network loss for three types of network models
Inset shows same data with x-axis in log scale
e Energy dissipated in updating the network weights for three types of network models
Inset shows same data with X-axis in log scale
it is possible that FN-DAM implementations or ML processors can naturally implement annealing without dissipating any additional energy
If such dynamics were to be emulated on other analog memories
it would require additional hardware and control circuitry
A mechanism to improve the dynamic range and the measurement resolution is to use a current-mode readout integrated with current-mode neural network architecture
If the read-out transistor is biased in weak-inversion
120 dB of dynamic range could be potentially achieved
the resolution of the weight would still be limited by the number of electrons and the quantization due to electron transport
Addressing this limitation would be a part of future research
due to the presence of thermal noise is in fact beneficial for training the neural network since it helps in overcoming artifacts due to local minima
Once the FN-DAM has transitioned to a non-volatile state (during inference)
the effect of blurring is significantly reduced as the energy barrier separating different analog states is significantly higher than energy due to thermal fluctuations
the effect of blurring due to measurement noise needs to be compensated by averaging or increasing the cumulative measurement time
the change in stored voltage can be coarsely adjusted
this would limit the number of updates before the weights saturate
Note that due to device mismatch the programmed values would be different on different FN-DAM devices
Since FN-DAM can also be implemented on conventional FLASH memories
the synapses could be scaled to future 3-D and 2.5D FLASH processes where high synaptic densities can be achieved for implementation of large-scale neural networks
a parameter that is very well controlled across processes
An oxide thickness greater than 10 nm ensures that the electron-leakage mechanism is dominated by FN quantum tunneling (instead of direct quantum tunneling)
FN-DAM devices should be implementable on most sub-10nm CMOS processes that allow fabrication of thicker gate-oxide transistors for input/output devices
FN-DAM exhibits a significant advantage compared to other analog memories
we have a used a hybrid chip-in-the-loop training paradigm
it is anticipated that in the future the training circuits and FN-DAM modules could be integrated together on-chip
The adaptability of the proposed FN-DAM could be used to mimic this effect in artificial machine learning systems
Exploiting this feature of the FN-DAM to mimic neurobiologically relevant synaptic dynamics in artificial neural networks would also be a topic of future research
input tunneling voltage was set to 21.5 V for 1 min and then the floating gate was allowed to discharge naturally
Readout voltages for the SET and RESET nodes were measured every 500 ms
The rate of discharge for each node was calculated
and a state where the tunneling rates would be equal was chosen as the initial synchronization point for the remainder of the experiments
The FN tunneling current is a function of the floating-gate capacitance CT and the floating-gate voltage V(t) and is given by:
where \({k}_{0}\) depends on initial condition as:
Many neural network training algorithms are based on solving an optimization problem of the form:27
where \(\bar{w}\) denotes the network synaptic weights
\({{{{{\mathcal{L}}}}}}(\cdot )\) is a loss-function based on the training set and \(\alpha\) is a hyper-parameter that controls the effect of the \({{{{{{\mathcal{L}}}}}}}_{2}\) regularization
Applying gradient descent updates on each element \({w}_{i}\) of the weight vector \(\bar{w}\) as:
Where the learning rate \({\eta }_{n}\) is chosen to vary according to \({\eta }_{n} \sim O(1/n)\) to ensure convergence to a local minimum:30
The naturally implemented weight decay dynamics in FN-DAM devices can be modeled by applying Kirchhoff’s Current Law at the SET and RESET floating gate nodes (see Fig. 1e)
Where \({C}_{{{{{{\rm{FG}}}}}}}+{C}_{C}={C}_{T}\) is the total capacitance at the floating gate
Taking the difference between the above two equations
For the differential architecture, \(w={W}_{S}-{W}_{R}\). Let \({{V}_{{{{{{\rm{train}}}}}}}=V}_{{{{{{\rm{SET}}}}}}}-{V}_{{{{{{\rm{RESET}}}}}}}\), the training voltage calculated by the training algorithm. In addition, \({I}_{{{{{{\rm{FN}}}}}}}\) is substituted from Eq. 2
Discretizing the update for a small time-interval \(\Delta t\)
Assuming that the stored weight (measured in mV) is much smaller than node potential (> 6 V) i.e.
\(w\ll {W}_{R}\) (and \({W}_{R}\approx {W}_{S}\)) and taking the limit \((\mu \to 1)\) using L’Hôpital’s rule:
\({W}_{S}\) follows the temporal dynamics given in Eq. 1
Comparing above equation to Eq. 4
the weight decay factor for FN-DAM system is given as:
the learning process is able to compensate for this deviation
A hybrid hardware-software system was implemented to carry out an online machine learning task
The physical weights (\(\bar{w}=[{w}_{1},{w}_{2}]\)) stored in two FN-DAM devices were measured and used to classify points from a labeled test data set in software
We sought to train a linear decision boundary of the form:
\(\bar{{{{{{\rm{x}}}}}}}=[{x}_{1},{x}_{2}]\) are the features of the training set
the error in the classification was calculated and a gradient of the loss function with respect to the weights was calculated
the weights were updated in hardware by application of SET and RESET pulses via a function generator
The states of the SET and RESET nodes were measured every 2 s and the weight of each memory cell
The factor of 1000 indicates that the weight is stored as the potential difference between the SET and RESET nodes as measured in mV
We followed a stochastic gradient descent method
The gradient of the loss function was calculated as:
Here \({\lambda }_{n}\) is the learning rate as set by the learning algorithm
The gradient information is used to update FN-DAM by applying control pulses to SET/RESET nodes via a suitable mapping function \(T\):
Positive weight updates were carried out by application of SET pulses and negative updates via RESET pulses
The magnitude of the update was implemented by modulating the number of input pulses
The parameter array \(K\) is estimated from experiments which were carried out at room temperature
Retention time \({T}_{{{{{{\rm{ret}}}}}}}\) is calculated by solving the following equation:
where \({W}_{R}\) is varied from 5.5 to 7 V, to simulate different operating regimes. These simulation results are shown in Supplementary Information Fig. 4a
The retention times could then be estimated at different operating temperatures by using the Arrhenius equation to estimate \({k}_{1}\) as a function of temperature as
These weights were then mapped back into the CNN
This learning process was carried on for 9 epochs
the weights were allowed to decay for the last epoch (note that in the standard CNN case
A special case with a 0.1% randomly assigned mismatch in the floating gate parameters (\({k}_{1}\) and \({k}_{2}\)) was also implemented
Neuromorphic computing using non-volatile memory
Analog architectures for neural network acceleration based on non-volatile memory
Memory devices and applications for in-memory computing
Hitting the memory wall: Implications of the obvious
Missing the memory wall: The case for processor/memory integration
In 23rd Annual International Symposium on Computer Architecture (ISCA’96) 90–90 (IEEE
In-memory computing with resistive switching devices
1.1 Computing’s energy problem (and what we can do about it)
In 2014 IEEE International Solid-State Circuits Conference Digest of Technical Papers (ISSCC) (IEEE
“Resistive random access memory (ReRAM) based on metal oxides
Magnetic tunnel junction based long-term short-term stochastic synapse for a spiking neural network with on-chip STDP learning
Recent progress in phase-change memory technology
Basic principles of STT-MRAM cell operation in memory arrays
Quantized conductance in ag/ges2/w conductive-bridge memory cells
High-performance mixed-signal neurocomputing with nanoscale floating-gate memory cell arrays
A FeFET based super-low-power ultra-fast embedded NVM technology for 22nm FDSOI and beyond
In 2017 IEEE International Electron Devices Meeting (IEDM) 19–7 (IEEE
Gu, X., Wan, Z. & Iyer, S. S. Charge-trap transistors for CMOS-only analog memory. In IEEE Transactions on Electron Devices Vol. 66, 4183–4187 https://doi.org/10.1109/TED.2019.2933484 (2019)
ECRAM as scalable synaptic cell for high-speed
In 2018 IEEE International Electron Devices Meeting (IEDM) 13–1 (IEEE
15.4 a 22 nm 2 mb ReRAM compute-in-memory macro with 121-28tops/w for multibit MAC computing for tiny AI edge devices
In 2020 IEEE International Solid-State Circuits Conference-(ISSCC) 244–246 (IEEE
A 1 mb 28 nm 1t1mtj STT-MRAM with single-cap offset-cancelled sense amplifier and in situ self-write-termination
Silicon support vector machine with on-line learning
In International Conference on Learning Representations (2019)
Fowler–Nordheim tunneling into thermally grown SiO2
Self-powered timekeeping and synchronization using Fowler–Nordheim tunneling-based floating-gate integrators
A self-powered analog sensor-data-logging device based on Fowler–Nordheim dynamical systems
An empirical model for device degradation due to hot-carrier injection
Learning on Silicon: Adaptive VLSI Neural Systems Vol
512 (Springer Science & Business Media
Pattern Recognition and Machine Learning (Springer
The use of multiple measurements in taxonomic problems
A practitioners’ guide to transfer learning for text classification using convolutional neural networks
In Proceedings of the 2018 SIAM International Conference on Data Mining 513–521 (SIAM
Numerical Optimization (Springer Science & Business Media
Rana: Towards efficient neural acceleration with refresh-optimized embedded dram
In 2018 ACM/IEEE 45th Annual International Symposium on Computer Architecture (ISCA) 340–352 (IEEE
Chaotic simulated annealing by a neural network model with transient chaos
Mastering the game of go with deep neural networks and tree search
Desynchronization of self-powered FN tunneling timers for trust verification of IoT supply-chain
Dadiannao: A machine-learning supercomputer
In 2014 47th Annual IEEE/ACM International Symposium on Microarchitecture 609–622 (IEEE
Deep learning with limited numerical precision
In International Conference on Machine Learning 1737–1746 (PMLR
temperature compensated CMOS floating-gate current memory with 130 ppm/K temperature sensitivity
Zhou, L. & Chakrabartty, S. A 7-transistor-per-cell, high-density analog storage array with 500 µV update accuracy and greater than 60 dB linearity. IEEE Symposium on Circuits and Systems (ISCAS 2014) https://doi.org/10.1109/ISCAS.2014.6865449 (2014)
Failure mechanisms of flash cell in program/erase cycling
In Proceedings of 1994 IEEE International Electron Devices Meeting 291–294 (IEEE
Regularization matters: Generalization and optimization of neural nets vs their induced kernel
In Advances in Neural Information Processing Systems (eds Wallach H
Sub-microwatt analog VLSI trainable pattern classifier
Equivalent-accuracy accelerated neural-network training using analogue memory
Metabolic costs and evolutionary implications of human brain development
Rahman, M. FNDAM_Nature_Data.zip. figshare. Dataset. https://doi.org/10.6084/m9.figshare.19295474.v1 (2022)
Download references
This work was supported in part by National Science Foundation (ECCS: 1935073)
Office of Naval Research (N00014-16-1-2426
and by National Institutes of Health (1R21EY028362-01)
These authors contributed equally: Darshit Mehta
designed the hardware and simulation experiments
designed the 24 element FN-DAM chipset; M.R
designed the 64 element FN-DAM chipset; D.M
conducted the simulation and hardware experiments; K.A
All authors contributed towards writing and proof-reading the manuscript
The authors declare no competing interests
Nature Communications thanks Subramanian Iyer and the other
Download citation
DOI: https://doi.org/10.1038/s41467-022-29320-6
Sign up for the Nature Briefing newsletter — what matters in science
Metrics details
a Principle of sensing and data logging where the input signal leaves its trace on a pair of synchronized dynamical system through a desynchronization process
b Equivalent circuit model of a self-powered dynamical system where the charge on a capacitor C stores the dynamical state of the system and the dynamics is governed by a leakage current I(Vt) and ambient stimuli xt
c Band diagram corresponding to the tunneling junction where the electrons tunnel across the triangular energy barrier and the input signal xt modulates the barrier shape
d Cross-section of the sensor-data-logging device showing the FN tunneling junction
the floating gate which is coupled to a read-out transistor P and a buffer B
with inset showing a pair of dynamical systems configured in a differential architecture
we showed that the continuous-time dynamics of this device can be modeled using a first-order differential equation which results in the change in floating-gate voltage Vt at time-instant t as
One of the FN device’s (labeled as the sensor) dynamics is modulated by an input signal xt
and its desynchronization is measured with respect to a reference FN device as:
For calculating Yt, we use the change from their initial voltages at time-instant t = 0 s (ΔVt in Fig. 2c) to eliminate the offset in the read-out stage.
a Equivalent circuit of the differential FN device coupled to the read-out circuitry
b Sensor and reference output voltages measured across nine trials after the device is initialized
c Change in sensor and reference values compared to the initial value V0 as ΔVt = Vt − V0
Shaded region in inset shows ±1 standard deviation
d Measured desynchronization between the sensing and reference devices
e Synchronization measured across a range of operating temperatures (5–40°)
The gradient (dark red to yellow) denotes an increase in operating temperature
f Standard deviation measured for the sensor
reference and the difference over 36 trials and across range of operating temperatures
we have derived a tractable mathematical model for the data sensed and stored by the sensor-data logger in response to an arbitrary time-varying input signal xt
We found that the output of the data logger YT measured at time-instant T can be expressed as
where Ax(T) represents the total “action” due to the input signal xt accumulated up to the time instant T and R(T) is a “forgetting” factor that is independent of the input signal xt
R(T) models the data retention capability and arises due to resynchronization of the sensor and reference FN devices
after the sensor device is perturbed by xt
we show that the action Ax(T) can be expressed in terms of device parameters as
and the resynchronization term R(T) can be expressed as
we show the “action” Ax(T) corresponding to different signal types with different magnitude and energy
The results show that Ax(T) is monotonic with respect to energy and hence can be used as a measure of cumulative energy
a Output measured from the device when subjected to an input pulse
During the positive half of the input pulse
the tunneling rate increases and desynchronizes the sensor device with respect to the reference device
b Responses measured from three loggers across three trials
The loggers were initialized to different conditions
hence the difference in their measured responses
c Sensor responses for input signals over a range of amplitudes
which can be accurately modeled by the action model and an ODE solver
The action model fits the data for this wide range of input conditions with an R2 of 0.9855
a Power to the system is switched off at the 1-h mark
The input pulse is applied at the 1.5-h mark for a duration of 120 s
c Output measured from the recorder when the power is ON and the comparison with the predicted model showing the process of desynchronization
d Recorder responses for input signals over a range of amplitudes
e Recorder responses for varying number of pulses (400 mV magnitude
f Distribution of absolute errors between measured data and model predictions
for externally powered and self-powered cases estimated across all experiments
we estimate the energy budget when the proposed sensor-data logger is driven by an arbitrary sensor signal
Noise in the system can also be described by the effective number of bits (ENOB) (Supplementary Fig. 8c)
10 bits precision can be initially expected in a system with 10 μV readout noise
ENOB would drop to 0 at ≈2 × 106 s (total recorder lifetime)
but with the added operational noise it takes ≈3 × 105 s to reach 0
Readings from multiple recorders can be combined to increase the ENOB of the system
a Experimental setup showing a piezoelectric (PVDF) transducer connected to the FN sensor-data-logger chipset
b Logger response when 58.6 mg (0.57 m/s2) acceleration was applied to the piezo cantilever (gain of 6\(\frac{{\rm{V}}}{{\rm{g}}}\) at 75 Hz resonant frequency) at 72 Hz for 100 s
c Recorder responses at different readout times for a range of input frequencies
All modulated responses were statistically different from the unmodulated case at all readout times
d Recorder was powered off in the shaded region
which was recorded as evidenced by the recorder value when power supply was turned on
this effect is weaker than that of other factors
when the magnitude of the input signal is 100 mV
15 fJ is used for charging a 300-fF input capacitor
The DC input impedance of the proposed device was measured to be >1017 Ω; thus
the energy required to maintain a voltage potential of 100 mV for 120 s is <100 aJ
Many signals of interest have power levels greater than this
and can provide sufficient energy for modulating the sensor
provided the system impedance is matched to the source
we show that a broadband AC input signal with upper cutoff frequency of 1 kHz and amplitude of 100 mV has an estimated energy dissipation by the system of 5 aJ for an event lasting 100 s
Using FN quantum tunneling to implement the dynamical system has some key advantages
Its stability allowed us to create a pair of synchronized devices which is compensated for environmental variations
and we were able to derive a recorder response model that matched experimental data with 98.8% accuracy
Its dynamics follow a \(1/\mathrm{log}\,(t)\) characteristic
The non-linear response leads to rectification of input signals and offers an opportunity for time stamping and reconstruction
A more rigorous and theoretic investigation into the use of dynamical systems for information reconstruction will be the topic of future research
while maintaining the capacitance ratio (CR)
an optimum balance between the input capacitor
decoupling capacitor and parasitic capacitance at the poly-substrate tunneling junction needs to be obtained
Better matching of the sensor and reference nodes (tunneling junctions
capacitors and readout circuits) using advanced analog layout techniques should be able to reduce the operational noise in the recorder and thereby increase the data retention capacity
Readout and common-mode noise can be further reduced by implementing a low-noise on-chip instrumentation amplifier
Multiple units of independent recorders could be used to increase the SNR of the recordings
we have described a self-powered sensor-data-logger device that records a cumulative measure of the sensor signal intensity over its entire duration
we designed a pair of synchronized dynamical systems whose trajectories are modulated by an external signal
The modulation leaves its trace by desynchronizing one of the synchronized pairs
The total cumulative measure or action is stored as a dynamical state which is then measured at a later instant of time
The self-powered dynamical system was designed by exploiting the physics of FN quantum tunneling in floating-gate transistors
We modeled the response of our system to an arbitrary signal and verified the model experimentally
We also demonstrated the self-powered sensing capabilities of our device by logging mechanical vibration signals produced by a small piezoelectric transducer
while being disconnected from any external power source
the readout voltage was programmed to around 3 V while the tunneling node was operating in the tunneling regime
This was achieved through a combination of tunneling and injection
input pins to 5 V and the program tunneling pin (Vprog) was gradually increased to 23 V
the tunneling node’s potential would start increasing
The coupled readout node’s potential would also increase
When the readout potential went over 4.5 V
electrons would start injecting into the readout floating gate
thus ensuring its potential was clamped below 5 V
VDD was set to 5 V for the rest of the experiments
Vprog to 21.5 V for 1 min and then the floating gate was allowed to discharge naturally
Readout voltages for the sensor and reference nodes were measured every 30 s
The rate of discharge for each node was calculated; and a state where the tunneling rates would be equal was chosen as the initial synchronization point for the remainder of the experiments
floating gates were initialized to the initial synchronization point
This was done by either setting the input to stable DC point through a digital to analog converter (DAC) or if the DAC value needed was beyond its output limit
then the potential would be increased by setting Vprog pin to 21 V
FN tunneling current density JFN across a triangular barrier can be expressed as a function of the electric field E across the barrier19:
where α and β are process and device specific parameters19
for a tunneling junction with cross-sectional area A and thickness tox
the tunneling current IFN for a time-varying voltage Vt is given by
Referring to the equivalent circuit in Fig. 2a
the dynamical system model when the sensing signal xt is absent is given by
where Ctotal = C + Cin is the total capacitance at the floating-gate node
The solution of the equation can be expressed as :
depend on material properties and device structure
Desynchronization between the sensor and reference nodes shown in Fig. 2a occurs because of differences in rates of tunneling
which are caused by differences in electric potentials across the respective floating gates
The reference node \({V}_{t}^{R}\) follows the dynamics of Eq. (10) as it is not under the action of an external field
The potential across the sensing node is given by how much it has desynchronized from the reference node (\({V}_{t}^{R}-{Y}_{t}\)) and the effect of the external field
where CR is the coupling ratio due to capacitive divider formed by Cin and Cfg
Substituting \({V}_{t}^{R}\) and \({V}_{t}^{R}\) in Eq. (13)
The above equation is the constitutive differential equation and can be solved using numerical methods for any input signal
To obtain an explicit expression for estimating the response Yt
we assume that Yt ≪ Vt and E(xt) = 0 for all t
and use Taylor series expansion with first-order approximation
Integrating both sides with respect to dt between the limits 0 and T:
Substituting f(Vt) from Eq. (12) into Eq. (19)
Further information on research design is available in the Nature Research Reporting Summary linked to this article
Gaas solar cells for indoor light harvesting
In 2014 IEEE 40th Photovoltaic Specialist Conference (PVSC)
Piezoelectric nanogenerators based on zinc oxide nanowire arrays
Self-powered autonomous wireless sensor node using vibration energy harvesting
An asynchronous analog self-powered cmos sensor-data-logger with a 13.56 mhz rf programming interface
A 5 nw quasi-linear cmos hot-electron injector for self-powered monitoring of biomechanical strain variations
A battery-less thermoelectric energy harvesting interface circuit with 35 mv startup voltage
Energy extraction from the biologic battery in the inner ear
Texas Instruments. bq25505 - ultra low-power boost charger with battery management and autonomous power multiplexer for primary battery in energy harvester applications. SLUSBJ3F. http://www.ti.com/lit/ds/symlink/bq25505.pdf
Information processing using a single dynamical node as complex system
Information processing capacity of dynamical systems
Mehta, D., Raman, B. & Chakrabartty, S. Differential Fowler-Nordheim tunneling dynamical system for attojoule sensing and recording. In 2019 IEEE International Symposium on Circuits and Systems (ISCAS) 1–5 https://doi.org/10.1109/ISCAS.2019.8702685 (IEEE
Fowler-Nordheim tunneling into thermally grown sio2
A cmos programmable analog memory-cell array using floating-gate circuits
On the effectiveness of vibration-based energy harvesting
Celebration of the tenth transducers conference: the past
present and future of transducer research and development
Thirty years of isfetology: what happened in the past 30 years and what may happen in the next 30 years
Organic field-effect transistor sensors: a tutorial review
Smart dust: communicating with a cubic-millimeter computer
Wireless recording in the peripheral nervous system with ultrasonic neural dust
Mehta, D. & Chakrabartty, S. Self-powered analog sensor-data-logger experimental data. https://doi.org/10.6084/m9.figshare.12814592.v1 (2020)
Download references
This work was supported in part by NIH research grants 1R21EY028362-01 and 1R21AR075242-01
The authors acknowledge the help and resources provided by Prof
Srikanth Singamaneni and Prashant Gupta in acquiring micrographs of the fabricated chips
Liang Zhou for useful discussions regarding quantum tunneling dynamics and circuit design
Owen Pochettino is acknowledged for helping build a chip testing station
developed the instrumentation for data collection
performed the experiments and collected the data
analyzed the data and generated the figures
All authors contributed to the writing of the manuscript
Peer review information Nature Communications thanks Michel Maharbiz and the other
anonymous reviewer(s) for their contribution to the peer review of this work
Download citation
DOI: https://doi.org/10.1038/s41467-020-19292-w
State regulators on Monday released their draft rules for what to do with all the hazardous oilfield waste that’s left over once a well is drilled
The announcement gives the public one month to comment on the new rules — while some industry representatives started giving input more than two years ago
Oilfield waste executives and consultants helped write the regulations beginning in 2021
Oil and gas business advocates also gave feedback to the Railroad Commission of Texas
The effort was initiated by a commissioner who has investments in oilfield waste companies
one of the agency’s three elected commissioners
ran for his seat with an eye on rewriting what’s known as Rule 8
Wright owns stock in several hazardous waste management companies in Texas
according to statements filed with the Texas Ethics Commission
Wright brushed off critics who suggest his involvement in the industry makes him a biased regulator
He said that he had little to do with re-writing the rules after he became commissioner
his position on the Commission has hurt his businesses rather than helped it
Few companies want to risk doing business with companies associated with regulators
“For those who think this is my rule — what Jim Wright wants — that couldn’t be further from the truth,” Wright said
[commission] staff knew we really needed to take a hard look at Rule 8.”
Wright said he believes the new rules will benefit all Texans
Supporters of industry’s early involvement say the rules, which haven’t been significantly revised since 1984
needed to be changed to make the permitting process more efficient and to allow new waste recycling technologies to be permitted
Critics say the revised regulations would benefit the industry over the public
“There’s an obvious conflict of interest if the industry gets to rewrite their own rules to their own financial benefit
and they end up writing rules that make people sick or contaminate groundwater and put our collective future at risk,” said Virginia Palacios
a watchdog group that advocates for stricter financial policies for commissioners
who does communications and government affairs for the Permian Basin Petroleum Association
which provided input on the draft rules to the Commission before they were released
“With all due respect to our friends on the environmental NGO side
they don't know what the field application is; they don’t understand what operators are literally doing day in and day out,” he said
“We all want robust environmental standards.”
Railroad Commission spokesperson Patty Ramon said soliciting very early industry input is typical for the agency's rulemaking process
Ramon said that at least one member of the public who had protested a facility’s permit in the past was also invited to provide early feedback
The obscure rules govern the disposal of massive amounts of waste
Companies drill thousands of wells every year in Texas
They typically pump mud into the ground as they drill; rocky soil and a salty liquid known as “produced water” then comes up along with the oil and natural gas
The Railroad Commission uses Rule 8 to decide how companies should handle that material. Unlike most hazardous waste, the toxic muck from the oilfield is exempt from federal regulations
The state regulations govern how waste can be recycled or dumped — typically in pits near the well or in commercial hazardous waste pits
The pits can leak toxic chemicals and radioactive materials and pollute surface or groundwater if not properly managed
In recycling, the mud can be cleaned and used for more drilling, rocks and gravel can be used to build roads and some of the less-contaminated water can be removed for other uses. However, “produced water” is most often injected back into the earth under a different permit, a method that has caused an increase in earthquakes across West Texas
The rule change would impose new environmental standards such as restricting where waste pits can be located; allow companies to suggest new forms of oilfield waste recycling; and limit who can protest permits
which environmental groups warn could limit public input
Ramon wrote that filing a protest is “not a cumbersome process” and that the changes would prevent competitors from filing protests
There will then be another formal proposal and chance for comment later
Texans for years have tried to stop oilfield waste dumps from moving into their communities — a fight that some say is already an uphill battle
drivers haul waste to a commercial pit facility next to 63-year-old Ron Pilsner’s family’s farm
Pilsner says the facility ruined their sense of peace: Bright lights shine from it at night
There’s constant beeping from vehicles backing up and often the wafting stink of petroleum
insecticides and what he describes as a smell like skunks
He no longer wants to open the windows and he worries about the waste pits' liners leaking and contaminating the area’s groundwater
The agency approved it anyway; a lawsuit by residents seeking to overturn the decision failed
After Petro Waste Environmental began construction and operations
the nuisance grew bad enough that Pilsner’s dad stopped renovating the farmhouse
He moved into a nursing home before he ever got to sleep on the new mattresses
Pilsner toured the waste pit’s perimeter with Sister Elizabeth Riebschlaeger
an 87-year-old Catholic nun who had family who lived in Nordheim and who supported the residents in their fight
Riebschlaeger argued the commission needed to give citizens more of a say
“Of course we’re defeated,” Riebschlaeger said
said it was in compliance with the current Rule 8 and did not expect to need to make any changes based on the draft rules
The company said it did stop accepting some materials in 2021 that smell and was investing in reducing truck traffic at the facility
safety is a core value and we are committed to being a good neighbor,” the statement said
only people like the Pilsners who own land adjacent to a proposed waste pit or recycling facility would be notified of a company’s intent to locate its facility there
And only people who can prove they would suffer “actual injury or economic damage” from a waste pit would be allowed to protest a new facility permit — a definition that would limit environmental groups’ influence in stopping new pits from being built
Those people would have 15 days to file a protest
from the time the company filed the application or last provided public notice
and the company would then have 30 days to either withdraw its permit application or request an administrative hearing to settle the dispute
The draft rules also introduce an option for companies to create pilot programs for their waste: Instead of dumping it in pits or recycling it
companies could propose alternative recycling methods not covered by the rules
The change addresses the industry’s concern that the current regulations aren’t flexible enough to include new technologies
But environmental groups worry that new methods could get a fast-track to permits with little oversight
such pits also can’t be located on a beach
Nor can they be located within 500 feet of any public water system well or intake location
The old rules said liners for waste pits must “reasonably” prevent pollution but didn’t include specific standards
The draft rules say pits must be lined with a plastic strong enough to resist damage from crude oil
Critics of the commission said the new liner standards aren’t much stronger than the internal guidance used by the agency
Critics also point out that the draft rules don’t spell out the penalties when pits leak or operators violate the rules of their permit
said that more details on fines would be available in the formal rule proposal and would likely be similar to existing regulations
Fines can be determined on a case-by-case basis and could be reduced if a company demonstrates “good faith;” critics say that would give companies more wiggle room to contest fines
The draft rules fulfill a goal and campaign promise for Wright
a Republican from South Texas who was elected to the Railroad Commission in 2020
Wright first tried to influence the agency's regulations years ago
when he was part of the oilfield waste services industry
Wright was the CEO and president of a Corpus Christi company called Environmental Evolutions
and has investments in other hazardous waste companies
Wright wanted to help guide the commission’s staff on how to more consistently apply the regulations affecting them
At the time, one commissioner agreed to give the group access to commission staff members, according to an interview Wright did on a podcast, but none of the staff actually wanted to work with them on the rules at that time. A 2019 bill to formalize a commission-appointed oil and gas advisory group failed to pass
So Wright decided to run for a seat on the Railroad Commission
Wright received campaign donations from the oilfield waste industry
the oilfield waste division for Tulsa-based NGL Energy Partners
is one of Wright’s top donors and has given him $226,000 since 2019; a company executive gave an additional $2,500
The company has also donated to the campaigns of the other two commissioners
Wright said that campaign fundraising was a “necessary evil” to be in politics
but that campaign donations don’t impact his decisions on the Railroad Commission and that he makes that clear to donors
After he defeated the better-funded incumbent Ryan Sitton in an upset
An investigative watchdog group called Documented obtained copies of the documents through public records requests and shared them with the Tribune
Wright’s former director of public affairs
helped facilitate the formation of a regulatory task force that included at least seven people from oil and gas and oilfield waste companies
including Pioneer Natural Resources and Waste Management
the task force went page-by-page through a years-old attempt to revise the rules
using it as a framework to define more clearly how permits can and can’t be approved
an environmental engineering consultant who chaired the task force
The task force then gave its proposal to the commission
Commission staff then invited powerful oil and gas lobbying groups to take part in an “informal review” of the task force’s recommendations
Representatives from major companies such as ExxonMobil
and Chevron were invited to attend commission meetings about the rules
Those companies and at least one lobbying group sent feedback and questions
a consultant and former Railroad Commission employee who chaired a regulatory committee for the Permian Basin Petroleum Association
sent an email in August 2022 to a commission staff member raising concerns that an oil waste company may have been trying to craft the rules to its benefit
“I want to make sure that the waste handlers are not using the Commission to further their business
said that while Wright had reactivated the task force and requested their input
he was not involved in the group’s deliberations or suggestions to agency staff
“The task force was helpful in getting the proverbial rulemaking ball rolling,” Krejci wrote in an email
“The rule which was just released is not a product of the task force
but rather the Commission staff who have been working internally on these updates for quite some time.”
And Wright said that if the regulations were simply to benefit the waste management industry
they wouldn’t change at all — the status quo is almost always better for business
he characterizes the draft rules as a step forward in the Railroad Commission’s ability to better regulate an industry that’s dramatically changed over the last four decades and protect water resources from pollution
He points out that the rules include new setbacks from surface water and better standards for lining waste pits
“I don’t see [how this rule] was formulated for the benefit of industry at all.”
Disclosure: Exxon Mobil Corporation and Permian Basin Petroleum Association have been financial supporters of The Texas Tribune, a nonprofit, nonpartisan news organization that is funded in part by donations from members, foundations and corporate sponsors. Financial supporters play no role in the Tribune's journalism. Find a complete list of them here
Choose an amount or learn more about membership
A tiny South Texas town is continuing to fight plans for an oil and gas waste site half its size
even after state regulators gave developers the go-ahead to build it.
A citizen's group in Nordheim — population 316 at last count — is suing the Texas Railroad Commission
challenging the petroleum regulator’s decision to permit a facility that would store waste including drill cuttings
fracking sand and other toxic oilfield leftovers
Filed late last month in Travis County district court
the lawsuit argues that the commission's three members erred in May when they unanimously approved the development by San Antonio-based Pyote Reclamation Systems.
It’s the DeWitt County community’s last-ditch effort to thwart the site, prolonging one of the first organized protests against industry activity in South Texas’ Eagle Ford Shale. Residents say the project threatens their way of life
“I’m hoping it’ll stop it,” said Paul Baumann
a retiree of the DuPont chemical company who owns ranch land bordering the waste site
“I was happy to see we had another option
I think the railroad commission was wrong.”
said she could not comment on pending litigation but that “it is important to know that protection of public safety and natural resources is our highest priority.”
Nordheim’s yearslong protest has gained attention in state energy circles
if only because of residents’ persistence.
It has also highlighted gaps in bureaucracy that prevented the commission
charged only with evaluating groundwater effects
from taking into account residents’ other quality-of-life concerns
Those include the site’s possible foul smell
new trucks expected to rumble down already cracked local roads and the facility’s proximity to a school
The waste site would include a mix of lined disposal pits and land treatment cells where more benign waste would be scattered and allowed to mix with soil
Pyote says thick clay lies beneath the land
providing an extra layer of protection for the groundwater
more than 200 people — including local state lawmakers and DeWitt County Judge Daryl Fowler — have asked regulators to reject the permit
Several times over the saga, dozens of residents — often wearing matching t-shirts — drove two hours north to Austin to voice their opposition before the commission. At one meeting
that included most of the Nordheim High Pirates senior class
When the commissioners finally granted approval, Commissioner Ryan Sitton said he did not like the site
But he had no other choice but to approve it
because experts at the agency determined that safeguards at the facility would properly protect groundwater in the area and because they could not evaluate other concerns
“We have taken great lengths to ensure our company has designed a facility that exceeds all of the regulatory requirements
and the Railroad Commission has ruled that we have done so,” he said Tuesday
“We’re very confident that the district court will uphold that ruling.”
The 16-page lawsuit alleges that the commission erred in several ways when granting the permit — including by allowing Pyote to revise its application multiple times over the permitting process
Allowing too many supplements violates agency rules
an environmental lawyer who has been involved in cases concerning other solid waste facilities
said Pyote’s “fairly extraordinary set of amendments and supplements” suggested that its application was “a pretty big mess.”
he said. “They have a slim chance of getting something based on procedural irregularities
but this is an administrative appeal of an agency decision
which carries with it a very powerful standard of deference to the agency."
Baumann said he was optimistic that the judge would side with his community
though he admitted he's not schooled in the case's finer legal points
“It’s just something to see how our legal system works,” he said
Choose an amount or learn more about membership.
A prototype FN-synapse array was fabricated in a standard silicon process and was used to verify the optimal memory consolidation characteristics and used for estimating the parameters of an FN-synapse analytical model. The analytical model was then used for large-scale memory consolidation and continual learning experiments.
We show that compared to other physical implementations of synapses for memory consolidation, the operation of the FN-synapse is near-optimal in terms of the synaptic lifetime and the consolidation properties. We also demonstrate that a network comprising FN-synapses outperforms a comparable elastic weight consolidation (EWC) network for some benchmark continual learning tasks.
With an energy footprint of femtojoules per synaptic update, we believe that the proposed FN-synapse provides an ultra-energy-efficient approach for implementing both synaptic memory consolidation and continual learning on a physical device.
Volume 16 - 2022 | https://doi.org/10.3389/fnins.2022.1050585
This article is part of the Research TopicPhysical Neuromorphic Computing and its Industrial ApplicationsView all 10 articles
Introduction: For artificial synapses whose strengths are assumed to be bounded and can only be updated with finite precision
achieving optimal memory consolidation using primitives from classical physics leads to synaptic models that are too complex to be scaled in-silico
Here we report that a relatively simple differential device that operates using the physics of Fowler-Nordheim (FN) quantum-mechanical tunneling can achieve tunable memory consolidation characteristics with different plasticity-stability trade-offs
Methods: A prototype FN-synapse array was fabricated in a standard silicon process and was used to verify the optimal memory consolidation characteristics and used for estimating the parameters of an FN-synapse analytical model
The analytical model was then used for large-scale memory consolidation and continual learning experiments
Results: We show that compared to other physical implementations of synapses for memory consolidation
the operation of the FN-synapse is near-optimal in terms of the synaptic lifetime and the consolidation properties
We also demonstrate that a network comprising FN-synapses outperforms a comparable elastic weight consolidation (EWC) network for some benchmark continual learning tasks
Discussions: With an energy footprint of femtojoules per synaptic update
we believe that the proposed FN-synapse provides an ultra-energy-efficient approach for implementing both synaptic memory consolidation and continual learning on a physical device
Two reservoirs with fluid levels W+ and W− are coupled to each other using a sliding barrier X
The barrier is used to control the fluid flow from the respective reservoirs into an external medium
which are modeled by functions J(W+) and J(W−)
at time-instant t are modulated by the position of the sliding barrier X(t) and the level of fluid in the external reservoir m(t)
the synaptic weight is stored as Wd=12(W+-W-) whereas Wc=12(W++W-) serves as an indicator of synaptic usage with respect to time
Figure 1. On-device memory consolidation using FN-synapses: (A) An illustration of a biological synapse with different coupled biochemical processes that determine synaptic dynamics (B) physical realization of the cascade model reported in Benna and Fusi (2016) that captures the consolidation dynamics using fluid in reservoirs uk that are coupled through parameters gkj
(C) Illustration of the FN-synapse dynamics using a differential reservoir model and its state at time-instants t0
and t2; (D) energy-band diagram to show the implementation of the reservoir model in (C) using the physics of Fowler-Nordheim quantum-mechanical tunneling where a single synaptic element (as show in E) which stores the weight Wd as the differential charge stored between each tunneling junction
Wd=W+-W-2 and the common-mode tunneling voltage Wc as the average of the individual charges
Wc=W++W-2); (E) micrograph of a single FN-synapse; (F) micrograph of an array of FN-synaptic devices fabricated in a standard silicon process
we show that for a synapse based on a general differential reservoir model [without making assumptions on the nature of the flow function J(.)] the synaptic weight Wd evolves in response to the external input X(t) according to the coupled differential equation
is a time varying decay function that models the dynamics of the synaptic plasticity as a function of the history of synaptic activity (or its usage)
The usage parameter Wc evolves according to
We will show in the Section 2 that m(t) can be used to tune the memory consolidation characteristics of the FN-synapse array to achieve memory capacity similar to or better than the cascade consolidation models (with different degrees of complexities) or the task-specific synaptic consolidation corresponding to the EWC model
Experimental weight evolution of FN-synapse: (A) A random set of potentiation and depression pulses of equal magnitude and duration applied to the FN-synapse leading to (B) bidirectional evolution of weight (Wd) and (C) the corresponding trajectory followed by the common-mode tunneling node (Wc)
Experimental characterization of a single FN-synapse: (A) Dependence of change in magnitude of weight with change in pulse-width which follows a linear trajectory defined by y = mx + c (where m = 0.005136 and c = −6.227 × 10−5)
(B) Dependence on pulse magnitude of the input pulse which follows an exponential trajectory defined by y = c × exp(ax + b) + d (where a = 1
(C) Change in the magnitude of successive weight updates (ΔWd) corresponding to repeated stimulus
If n denotes the number of new memory patterns that have been applied to an empty FN-synapse array (initial weight stored on the network is zero)
then the Section 3 shows that for the pth update the retrieval memory signal S(n
where γ>0 is a device parameter that depends on the initialization condition
material properties and duration of the input stimuli
(F) SNR comparison of the γ1 and γ2 models with the analytical model for 1,000 Monte Carlo simulations
The legends associated with the plots are specified as (γ
All of these results correspond to the behavior of an empty FN-synapse network
This implies that the synapses have become rigid with an increase in its usage
This type of memory consolidation is also observed in EWC models which has been used for continual learning
note that unlike EWC models that need to store and update some measure of Fisher information
here the physics of the FN-synapse device itself can achieve similar memory consolidation without any additional computation
in the case for FN-synapse under m1(t) and m2(t) modulation profile the #patterns.retained reaches a finite value similar to that of the cascade models
This indicates that the FN-synapse network when subjected to plasticity modulation profiles continues to form new memory while gracefully forgetting the old ones
For the m3(t) modulation profile the network is slowly evolving and yet to reach the steady state condition within 2000th update
The FN-synapse network under the m4(t) modulation profile
which switches between m0(t) and m1(t) periodically
is in an oscillatory steady-state with the same periodicity as the modulation profile itself
note that the network does not suffer from blackout catastrophe and has a variable capacity
This shows that the capacity of the FN-synapse network can also be tuned to the specificity of different applications
we also observe that the steady state network capacity for m2(t) modulation profile is higher than that of cascade models
Note here that network capacity for cascade models may be increased by increasing the complexities of the synaptic model
we find that network capacity for FN-synapse is comparable to cascade models of moderate complexities
Network capacity and saturation experiments: Comparison of (A) no
of patterns retained by networks composed of 1,000 synapses following different synaptic models when exposed to 2,000 patterns and (B) steady-state SNR of the 1000th update (p = 1
000) of networks consisting of 1,000 synapses with various synaptic models when exposed to subsequent updates
For m4(t) modulation SNR profiles for both 450th and 1000th (p= 450,1000) updates are shown
our synaptic models can exhibit memory consolidation properties similar to both EWC and steady-state models while being physically realizable and scalable for large networks
The next set of experiments was designed to evaluate the performance of FN-synapse neural network for a benchmark continual learning task. A fully-connected neural network with two hidden layers was trained sequentially on multiple supervised learning tasks. Details of the neural network architecture and training are given in Section 3 and in the Supplementary material
The network was trained on each task for a fixed number of epochs and after the completion of its training on a particular task tn
the dataset from tn was not used for the successive task tn+1
The aforementioned tasks were constructed from the Modified National Institute of Standards and Technology (MNIST) dataset, to address the problem of classifying handwritten digits in accordance with schemes popularly used in several continual-learning literature (Hsu et al., 2018)
Also known as incremental domain learning using split-MNIST dataset
each task of this continual learning benchmark dictates the neural network to be trained as binary classifier which distinguishes between a set of two hand-written digits
the network is first trained to distinguish between the set [0
1] as t1 and is then trained to distinguish between [2
the network acts as an even-odd number classifier during every task
it is clearly evident that both the FN-synapse networks significantly outperform the others
It is also worth noting here that even when a network equipped with FN-synapse is trained using a computationally-inexpensive optimizer such as SGD
it shows remarkably superior performance than highly computationally-expensive approaches such as ADAM with conventional memory and ADAM with EWC variants
Continual learning benchmarks results and insights: (A) Overall average accuracy comparison of SGD and ADAM with FN-synapse
(B) Distribution of the usage profile of weights in the output layer and the input layer of the FN-synapse neural network
Overall Average Accuracy comparison of incremental-domain learning scenarios on the Permuted MNIST dataset using (C) ADAM with EWC
ADAM with FN-Synapse and ADAM with conventional memory and (D) ADAGRAD with conventional memory and ADAGRAD with FN-synapse
we show that the ability of the network to learn or forget new tasks is a function of the initial plasticity of the FN-synapses and can be readily adjusted
The main methods are described in this section of the paper while Supplementary material includes additional details
Consider the differential synaptic model described by Figure 1C where the evolution of two dynamical systems with state variables W+ and W− is governed by
where J(.) is an arbitrary function of the state variables
+12X(t) or -12X(t) are differential time varying inputs and m(t) is a common mode modulation input
we define the weight parameter Wd as Wd=12(W+-W-) which represents the memory and the common-mode parameter Wc as Wc=12(W++W-) which represents the usage of the synapse
applying Taylor series expansion on (10) and (11) leads to
This means that the modulation input impacts the usage of the synapse
the plasticity of the synapse can be tuned using m(t) when needed
Now we first look into the trivial case when a constant modulation input is provided
m(t) = c where c is any arbitrary constant
In this scenario the plasticity of the synapse is solely dependent on the usage of the synapse as m(t) does not change with time
Substituting the derivative of Wc from (12)
the rate of change in Wd can be formulated as:
Please refer to the Supplementary material for detailed derivation
Equation (14) shows that the change in weight ΔWd is directly proportional to the curvature of usage while being inversely proportional to the rate of usage
comparing the weight update equation in (14) to the weight update equation for EWC in the balanced input scenario
the decay term has the following dependency with time for avoiding catastrophic forgetting
the usage of a synapse is always monotonically increasing and since Wc represents the usage
At the same time Wc also needs to be bounded
therefore Wc has to monotonically decrease with increasing usage while satisfying the relationship in Equation (16)
It can be shown that Equations (16) and (15) can be satisfied by any dynamical system of the form
where f(.) ≥ 0 is any monotonic function
Substituting Equation (17) in Equation (15) we obtain the corresponding usage profile as follows
where f′(logt) and f″(logt) are derivatives of f(logt) with respect to logt
While several choices of f(.) are possible
the simplest usage profile can be expressed as
The corresponding non-linear function in this model is determined by substituting Equation (19) in Equation (12) to obtain
The expression for J(.) in Equation (20) bears similarity with the form of FN quantum-tunneling current (Lenzlinger and Snow, 1969) and Figures 1D–F show the realization of Equations (6) and (7) using FN tunneling junctions
For the differential FN tunneling junctions shown in Figure 1F and its equivalent circuit shown in the Supplementary Figure 1
W− are the tunneling junction potentials
vin(t) is the input voltage to the coupling capacitance and CT = Cc + Cfg is the total capacitance comprising of the coupling capacitance and the floating-gate capacitance Cfg
J(.) are the FN tunneling currents given by
where k1 and k2 are device specific and fabrication specific parameters that remain relatively constant under isothermal conditions
Following the derivations in the previous sections and the expression in Equation (19) leads to a common-mode voltage Wc profile as
where k0=exp(k2Wc0) and Wc0 refers to the initial voltage at the floating-gate
Upon following the same procedure used in previous sections
the weight update equation for an FN-synapse using Equation (21) and Equation (22) can be expressed as
We designed the floating-gate potential and the input voltage pulses such that the FN-dynamics is only active when there is an memory update
the dynamics in Equation (26) evolve in a discrete manner with respect to the number of modulations
Assuming CT = Cc we formulate a discretized version of the weight update dynamics from Equation (26) in accordance with the floating-gate potential profile of the device expressed in Equation (25) as follows
where n represents the number of patterns observed and Δt is the duration of the input pulse
we obtain the weight update equation with respect to number of patterns observed as
the memory update can be expressed as a weighted sum over the past input as
We define the retrieval signal and the noise associated with it as per the definition in Benna and Fusi (2016)
each weight in the network is indexed as Wd(a
the input applied to the ath synapse after n patterns is vin(a
the signal strength for the pth update (where p < n) introduced to the initially empty network tracked after n patterns can be formulated as:
where angle brackets denote averaging over the ensemble of all of the input patterns seen by the network
If we assume that the input patterns are random binary events of ±1 and are uncorrelated between different synapses and memory patterns then substituting Equation (31) in Equation (32)
the term (1+2ln(k1△tn+k0))≈1
where γ=k0k1△t and depends on the pulse-width △t and the initial condition k0
The above equation shows that the signal's strength is a function of the system parameter γ and decays with the number of memory pattern observed
If we assume that the weight Wd(n) is uncorrelated from the input vin(n) and that the inputs vin(1)
...vin(n) are uncorrelated from each other
then the corresponding noise power is given by the variance of the retrieval signal expressed in Equation (32)
This can be estimated as the sum of the power of all signals tracked at n except for the retrieval signal corresponding to the pth update we are tracking and is given by:
in order to derive a more tractable analytical expression for further analysis we added the retrieval signal as well into the summation which introduces a small error in the estimation (overestimating the noise by the retrieval signal term)
This leads us to the following estimation of the noise power:
Based on the value of n in comparison to γ
we obtain two trends for the noise profile
which implies that noise increases with increase in updates initially
which implies that noise falls with increase in updates in the later stages
The signal-to-noise ratio (SNR) of a network of size N can then be obtained as:
we notice that without external modulation Wc decreases monotonically with each new updates which correspondingly makes the synapse only rigid
the idea is to keep Wc as steady as possible to keep the synapse plastic as long as possible by applying a modulation profile m(t) that recovers/restores Wc after every synaptic update
m(i) is the magnitude of the modulation increment
and T is the time between each modulation increment
This increment is determined by the rate of the differential update to the FN-synapse
Integrating this form of m(t) into Equation (12) leads to
which implies a tunable plasticity profile for the FN-synapse
An analytical solution to the differential equation (43) is difficult and hence we resort to a recursive solution
it can be seen that the initial condition of the variable Wc changes at increments of T
whereas between two modulation increments Wc evolves naturally according to Equation (25)
the dynamics of Wc in the presence of the modulation increments can be described as
where Vmod(t) is an external voltage signal applied to the FN-synapse as shown in Supplementary Figure 1 and is given by:
In this case the change in plasticity of the synapse is determined by the step-size of the staircase voltage function Vmod(t)
Note that the weight update equation in (13) is still valid since m(t) is kept constant during differential input
Although an analytic expression for the SNR is no longer tractable in this iterative form, the ability of the modulation term to regulate the plasticity and induce a more graceful form of forgetting is shown in the corresponding no. of patterns retained plot in Figure 5A and the SNR plot Figure 5B for various modulation input profiles
so here we describe the methods specific for this work
The tunneling node potential was initialized at a specific region where FN-tunneling only occurs while there is a voltage pulse at the input node and the rest of the time it behaves as a non-volatile memory
This was achieved by first measuring the readout voltage every 1 s for a period of 5 min to ensure that the floating gate was not discharging naturally
During this period the noise floor of the readout voltage was measured to be ≈100μV
an voltage pulse of magnitude 1 V and duration 1 ms was applied at the input node and the change in readout voltage was measured
If the change was within the noise floor of the readout voltage
the potential of the tunneling nodes were increased by pumping electrons out of the floating gate using the program tunneling pin
This process involves gradually increasing the voltage at the program tunneling pin to 20.5 V (either from external source or from on-chip charge pump)
The voltage at the program tunneling pin was held for a period of 30 s
The process was repeated until substantial change in the readout voltage was observed (≈300μV) after providing an input pulse
The readout voltage in this region was around 1.8 V
The fabricated prototype contained 128 differential FN tunneling junctions
due to the peripheral circuitry only one tunneling node could be accessed at a time for readout and modification
since the memory pattern is completely random
each synapse can be modified independently without affecting the outcome of the experiment
two tunneling nodes were initialized following the method described in the aforementioned section
Input pulses of magnitude 4 V and duration 100 ms was applied to both the tunneling nodes
The change in the readout voltages were measured
and the region where the update sizes of both the tunneling node would be equal was chosen as the initial zero memory point for the rest of the experiment
The nodes were then modified with a series of 100 potentiation and depression pulses of magnitude 4.5 V and duration 250 ms and the corresponding weights were recorded
This procedure represented the 100 updates of a single synapse
The tunneling nodes were then reinitialized to the zero memory point and the procedure was repeated with different random series of input pulses representing the modification of other 99 synapse in the network
The first input pulses of each series of modification forms the tracked memory pattern
To modify the value of γ the FN-synapses were initialized at a higher tunneling node potential
Adaption of FN-synapse occurs by tunneling of electrons through a triangular FN quantum-tunneling barrier
The tunneling current density is dependent on the barrier profile which in turn is a function of the floating-gate potential
W− is around 7 V the synaptic update ΔWd due to an external pulse can be determined by the continuous and deterministic form of the FN-synapse model (as described in the previous sections)
Since the number of electrons tunneling across the barrier is relatively large (≫1)
the method is adequate for determining ΔWd
each updates occurs due to the transport of a few electrons tunneling across the barrier and in the limit by a single electron tunneling across the barrier at a time
the continuous behavioral model is no longer valid
the behavioral model of the FN-synapse has to switch to a probabilistic model
we can assume that each electron tunneling event follows a Poisson process where the number of electrons e+(n)
e−(n) tunneling across the two junctions during the nth input pulse is estimated by sampling from a Poisson distribution with rate parameters λ+
A is the cross-sectional area of the tunneling junction
the corresponding discrete-time stochastic equation governing the dynamics of the tunneling node potentials W+(n)
where CT is the equivalent capacitance of the tunneling node
the SNR decay still obeys the power-law curve
in the form a multi-layered perceptron (MLP) with an input layer of 1024 nodes
two hidden layers of 400 nodes each (paired with the ReLU activation function) and a softmax output layer of 2 nodes
has been utilized by every method mentioned in this work
a learning rate of 0.001 was chosen for both SGD and ADAM (with additional parameters β1
Each model was trained with a mini-batch size of 128 for a period of 4 epochs
Similar to the continual learning experiments conducted on split-MNIST, benchmark incremental-domain learning experiments were also carried out by randomly permuting the order of pixels of the images in the MNIST dataset in accordance with Hsu et al. (2018) which is referred as the Permuted-MNIST
The architecture of the neural network employed is similar to the one for the split-MNIST with the exception of being equipped with 1,000 neurons in each of the two hidden layers instead of 400 and with 10 neurons in the output layer instead of 2
the network learns a new set of permutations of the 10 digits
The network was trained on 10 such tasks for 3 epochs using a learning rate of 0.0001 for ADAM and 0.001 for ADAGRAD
Corresponding to every weight/bias in the MLP
an instance of the FN-synapse model was created and initialized to a tunneling region according to the initial Wc value
ΔWd can be modulated linearly and precisely by changing the pulse-width of the potentiation/depression pulses
each weight update (calculated according to the optimizer in use) is mapped as an input pulse of proportional duration for the FN synapse instance
every instance of the FN-synapse model is updated according to Equation (27) and the Wd thus obtained in voltage is scaled back to a unit-less value and within the required range of the network
the synaptic updates require additional pre-processing of the gradients
which in some cases could be computationally and resource intensive
does not require any pre-processing of gradients and instead can exploit the physics of the device itself for synaptic intelligence and for continual learning
we have shown an FN-synapse network shows better multi-task accuracy compared to other continual learning approaches
This leads to the possibility that the intrinsic dynamics of the FN-synapse could provide important clues on how to improve the accuracy of other continual learning models as well
we show that FN-synapse based neural network is able to maintain its performance even when the network size is increased
it is possible that the network becomes capable of learning more complex tasks due to increase in overall plasticity of the network while ensuring considerably better retention than neural networks with traditional synapses
show that the number of patterns/memories retained in an FN-synapse network under modulation profile m2(t) at steady state is higher compared to that of a high-complexity cascade model for a network size of N = 1
Even though we have not used the interpolation feature for benchmark experiments
we believe that this attribute is going to provide significant improvements for continuous learning of a large number of tasks
The FN-synapse mimics the same regulatory mechanism through the decaying term r(t) that takes into account the history of usage or neuronal activity to determine the plasticity of the synapse for future use as well as prevents runaway effects by making the synapses rigid at saturation
the performance of the FN-synapse based neural network remains robust even in the presence of 5% device mismatch
achieving an optimal decay profile would require additional control circuitry
we believe that the FN-synapse represents one of the few
if not the only class of synaptic devices that can achieve optimal memory consolidation on a single device
The raw data supporting the conclusions of this article will be made available by the authors
SC and MR came up with the concept of FN-synapse
and SC designed the hardware and simulation experiments
MR designed the 64 element FN-synapse chipset
MR and SB conducted the simulation and hardware experiments
All authors contributed toward writing and proof-reading the manuscript
All authors contributed to the article and approved the submitted version
This work was supported in part by the National Science Foundation Grants FET: 2208770 and ECCS: 1935073
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations
Any product that may be evaluated in this article
or claim that may be made by its manufacturer
is not guaranteed or endorsed by the publisher
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fnins.2022.1050585/full#supplementary-material
Metaplasticity: tuning synapses and networks for plasticity
PubMed Abstract | CrossRef Full Text | Google Scholar
Metaplasticity: the plasticity of synaptic plasticity
PubMed Abstract | CrossRef Full Text | Google Scholar
Memory aware synapses: learning what (not) to forget
CrossRef Full Text | Google Scholar
Learning in neural networks with material synapses
CrossRef Full Text | Google Scholar
Computational principles of synaptic memory consolidation
PubMed Abstract | CrossRef Full Text | Google Scholar
Retrograde amnesia for spatial memory induced by NMDA receptor-mediated long-term potentiation
Riemannian walk for incremental learning: understanding forgetting and intransigence
CrossRef Full Text | Google Scholar
Accurate and efficient 2-bit quantized neural networks
Google Scholar
PACT: parameterized clipping activation for quantized neural networks
Deep direct reinforcement learning for financial signal representation and trading
Parallel programming of an ionic floating-gate memory array for scalable neuromorphic computing
Hebbian spike-driven synaptic plasticity for learning patterns of mean firing rates
PubMed Abstract | CrossRef Full Text | Google Scholar
Limits on the memory storage capacity of bounded synapses
PubMed Abstract | CrossRef Full Text | Google Scholar
Cascade models of synaptically stored memories
PubMed Abstract | CrossRef Full Text | Google Scholar
Palimpsest memories stored in memristive synapses
Customized binary and multi-level HfO2- x-based memristors tuned by oxidation conditions
Re-evaluating continual learning scenarios: a categorization and case for strong baselines
CrossRef Full Text | Google Scholar
Mechanisms of heterosynaptic metaplasticity
CrossRef Full Text | Google Scholar
Overcoming catastrophic forgetting in neural networks
Synaptic metaplasticity in binarized neural networks
Multi-level memristors based on two-dimensional electron gases in oxide heterostructures for high precision neuromorphic computing
Overcoming catastrophic forgetting by incremental moment matching
CrossRef Full Text | Google Scholar
Fowler–Nordheim tunneling into thermally grown SiO2
CrossRef Full Text | Google Scholar
Metaplasticity mechanisms restore plasticity and associativity in an animal model of Alzheimer's disease
Rotate your networks: better weight consolidation and less catastrophic forgetting
Intracellular calcium stores mediate metaplasticity at hippocampal dendritic spines
Memristors–from in-memory computing
and spiking neural networks to the future of neuromorphic and bio-inspired computing
A self-powered analog sensor-data-logging device based on Fowler-Nordheim dynamical systems
An adaptive synaptic array using Fowler–Nordheim dynamic analog memory
Design of memristor based low power and highly reliable reram cell
CrossRef Full Text | Google Scholar
Design of power-and variability-aware nonvolatile rRAM cell using memristor as a memory element
CrossRef Full Text | Google Scholar
Down-scalable and ultra-fast memristors with ultra-high density three-dimensional arrays of perovskite quantum wires
SpotKD: a protocol for symmetric key distribution over public channels using self-powered timekeeping devices
CrossRef Full Text | Google Scholar
Efficient partitioning of memory systems and its importance for memory consolidation
Low-memory neural network training: a technical report
CrossRef Full Text | Google Scholar
“Hybrid 8-bit floating point (HFP8) training and inference for deep neural networks
in Advances in Neural Information Processing Systems
Google Scholar
“Vanishing gradient mitigation with deep learning neural network optimization
in 2019 7th International Conference on Smart Computing & Communications (ICSCC) (IEEE)
CrossRef Full Text | Google Scholar
A multi-level memristor based on AL-doped HfO2 thin film
Sleep promotes branch-specific formation of dendritic spines after learning
Stably maintained dendritic spines are associated with lifelong memories
Improved multitask learning through synaptic intelligence
CrossRef Full Text | Google Scholar
Self-powered timekeeping and synchronization using Fowler-Nordheim tunneling-based floating-gate integrators
CrossRef Full Text | Google Scholar
Desynchronization of self-powered fn tunneling timers for trust verification of IoT supply chain
Bose S and Chakrabartty S (2023) On-device synaptic memory consolidation using Fowler-Nordheim quantum-tunneling
Received: 21 September 2022; Accepted: 28 December 2022; Published: 13 January 2023
Copyright © 2023 Rahman, Bose and Chakrabartty. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY)
distribution or reproduction in other forums is permitted
provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited
in accordance with accepted academic practice
distribution or reproduction is permitted which does not comply with these terms
*Correspondence: Shantanu Chakrabartty, c2hhbnRhbnVAd3VzdGwuZWR1
Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations
Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher
94% of researchers rate our articles as excellent or goodLearn more about the work of our research integrity team to safeguard the quality of each article we publish
roughly 10 percent of Nordheim residents (population 316 at last count) once again pulled on their yellow “Concerned About Pollution” T-shirts
drove two hours north to Austin and told Texas regulators that they did not want to live next to an oil and gas waste site roughly half their town’s size
Many in the rural DeWitt County community had done some version of this exercise several times over the past two or three years
They say their way of life would be threatened by a proposed 143-acre facility that would be used to store waste including drill cuttings
Most of the residents figured they’d return home officially defeated
The Texas Railroad Commission on Tuesday voted 3-0 to allow San Antonio-based Pyote Reclamation Systems to build the facility, effectively ending one of the first organized protests against industry activity in South Texas’ Eagle Ford Shale
“That’s what you call a little town getting shit on,” 80-year-old Kermit Koehler
the case highlighted gaps in bureaucracy that prevented the commission
charged with only evaluating groundwater effects
“That’s what you call a little town getting shit on.”— Kermit Koehler
“I’ll be candid — I don’t like the site,” said Commissioner Ryan Sitton
who met with Nordheim residents last fall in an unpublicized visit
He was the lone commissioner to weigh in at Tuesday's open meeting.
that he had no other choice because experts at the agency determined that safeguards at the facility would properly protect groundwater in the area and because they could not evaluate other concerns
The Texas Commission on Environmental Quality
has limited jurisdiction over such facilities
and only those with certain types of equipment require air permits to operate
No state or local agency has authority to address the land use and traffic concerns
The facility would border Nordheim but not sit within city limits
Read MoreDrilling Waste Site Roils Tiny Nordheim
Sitton issued an unusual warning: “Because they’ve chosen to build this site so closely to residents
the margin of error is exceptionally small,” he told an attorney for the waste company
CEO George Wommack told The Texas Tribune that his company’s facility is “best in class” and “the most highly engineered” such landfill that the commission has ever permitted.
“We care very much about these people,” he said
The commission ultimately required Pyote to beef up groundwater safeguards so that the waste won't flow out of retention ponds on site during worst-case flooding scenarios. Grant Chambless
who manages the agency’s environmental permits and support staff
said he "was very confident that the risks have been addressed."
Wommack said Pyote would iron out the additional quality-of-life concerns with local residents
“Now is the time that we can get some stuff figured out.”
That’s the type of waste Pyote’s facility would handle.
The facility would include a mix of lined disposal pits and land treatment cells where more benign waste would be scattered and allowed to mix with soil
The company says thick clay lies beneath the land
providing an extra layer of protection for the groundwater underneath. Wommack said the spot in question is ideal for waste disposal
It sits in prime Eagle Ford drilling acreage
And plenty of nearby drillers still need a spot to unload their waste — even amid the industry’s epic slowdown over the past two years
more than 200 people — including local state lawmakers and DeWitt County Judge Daryl Fowler — urged the commission to reject the permit
State Rep. Geanie Morrison
stood with her constituents throughout the long-winding protest
The waste site would “cause this community great problems,” she told the commissioners
she recognized the agency’s narrow jurisdiction and vowed to work during the 2017 legislative session on plugging such regulatory gaps
The Railroad Commission is already set to undergo particular scrutiny next year, as lawmakers consider recommendations from the state Sunset Advisory Commission.
“I want to offer to work with you during the sunset process,” she said to the commission
“Help me make this a better process.”
The acknowledgements and promises did not make Nordheim residents feel any better
“My roadrunners are going to be gone
and my brother and nephew [who live next to the site] are going to be gone,” Paul Baumann
a retiree of the DuPont chemical company who owns ranchland bordering the site
he told the Texas Tribune that he was “really disappointed.”
“Disgusted” is the word that Lynn Janssen kept using
Her ranch house sits just down a dirt road from the soon-to-be waste site
“To think that they can permit something that they don’t like
there’s something wrong with that.”
“People won’t send their kids to school there
The fracking boom has transformed vast parts of Texas
pumping money into local economies while raising fears about groundwater and air pollution
The boom has caused increased crime and truck traffic
But the oil and gas bonanza has also brought more subtle
we will examine how the oil economy has fractured families
If you’re in Nordheim on a Thursday morning
chances are you’re heading to Elo Pfeifer’s place on Broadway
That’s the day Pfeifer smokes the sausages that some say are the best for miles around
Pfeifer has been cooking barbecue and selling goods from his non-air-conditioned general store since he sold the bar across the road in the early 1970s
Broadway Grocery has now become something of an institution in this tiny South Texas outpost of about 300 residents
and the bar that has stood on the corner of Broadway and First Avenue since 1933 and a school that teaches through 12th grade
there isn’t much to Nordheim except for shuttered-up storefronts and a population in decline
Cuero attracts about $250,000 a year in sales taxes from local businesses and makes $30,000 a month selling its water to oil companies for use in the fracking extraction process
sits behind a large wooden meeting table flanked by U.S
she’s tied into nearly everything that happens in Nordheim
“People call me if the bank doesn’t open on time
Payne has a rolled-up blueprint of the town
“It looks like they were planning on the town expanding; were planning on it really growing,” she says
it’s almost impossible now to picture it as a town that once boasted 27 stores
five gas stations and a population double what it is today
a lot of kids stayed on at the farm after graduating,” Payne says
Cotton and corn were the biggest industries
and she remembers the backbreaking work in the fields: “All the kids would do it
We’d have a bag strapped to us and get 5 cents for 100 pounds of cotton
but it had to be delayed ’cause all of us had to go to work on the farm
Payne says Nordheim’s decline began in the 1950s when kids started going away to college
“I was born in 1936 and graduated in 1955 from Nordheim High School
A lot of girls went to San Antonio to business school or college.” She got married a year after she graduated and
I was working as a high school coach in Denver
and I later decided to move back to Nordheim,” she says
the town still has a strong community spirit
But that spirit won’t fix the roads or boost a town that remains stagnant while so many of its neighbors get rich
I learned that the roads had been torn up by oilfield trucks barreling through town day and night
Payne told me Cabeza Road was so bad that she’d had to shell out for caliche to fill in the holes (she couldn’t afford asphalt)
Then the town’s residents complained about all the dust the caliche was causing
The county eventually had to pay for resurfacing
Payne also refused to sell any of Nordheim’s water to fracking companies
She says she can’t preserve the water supply because other entities pull from the same aquifer
at least she can say she never sold any of it
But that decision also deprived the town of revenue
“It’s hard to see those other towns getting rich
She knows the roads in Nordheim need repairing
And she knows she needs to attract more businesses to town
could arrive if oil companies soon find oil under the town
“whatever money we get will go straight to our streets
but I’ll fix the streets before I fix this.”
“This fracking boom has been very good for us,” she says
“but it would be better if it would come under my town
the potential for profit comes with a cost
The engineering firm Pyote Reclamation Systems has proposed a disposal well for fracking waste in town that has enraged some Nordheim residents
The Texas Railroad Commission initially denied the proposal last year due to pollution concerns and protests by residents
The current application means fracking waste will evaporate from open
and allow 1,000 gallons per acre per day to leak from the collection pit
and 100 gallons per acre per day to leak from the trenches
Mayor Payne has become more receptive to the oil companies—a move that has alarmed some of her former champions
“I don’t think [the disposal well] should be that close to the city,” she tells me
“But if it’s put in right … controlled right … I’m neutral
Payne shows me a map of where the disposal site would be located
a quiet country lane on the outskirts of town
and she says each will have to be tested once a month
“but the lining of the pits are as thick as tires
Payne says they’re drilling so many wells nearby
“where do you expect them to put the waste?” And therein lies the rub
The Nordheim residents opposed to fracking because of environmental concerns think their mayor has turned against them
thinks the disposal well could be good for her town’s sagging economy
Margie Hull and her husband Patrick live across the road from the proposed disposal pit
They’ve lived in the same house for 30 years and run a real estate business
and Margie tells me she worries about the chemicals blowing toward the house
and the danger the truck traffic could pose to their animals
a retired school maintenance mangager who co-owns the local mercantile re-sale shop and is a volunteer firefighter
says the fracking boom has been good for Nordheim
most of our students that graduated from here would leave and just come back to visit their parents,” he says
You can live with the smell if you get all that money
but you can’t if you get sick all the time.”
Opposite the proposed disposal well site is a gate that opens onto sprawling fields; attached to the gate is a black-and-white sign of a skull and crossbones and the words “Don’t Dump on Nordheim.” Scrawled in handwriting next to it: “Proverbs 13:1—Heareth his Father’s instruction.” A similar sign a few feet farther down the fence reads: “A Christian would not impose this on his brothers and sisters.”
Kevin Styra—who was born 8 miles up the road from here and who now runs the farm that has been in his family for 118 years—put the signs up and wrote those messages to the oil companies
and to the landowner who leased them the land for the disposal wells
Styra would bale hay and help his uncle run cattle on the family’s land
Today he’s been out tilling the fields with the help of his 10-year-old daughter
while keeping a close eye on the number of trucks heading down the gravel road opposite his gate
Styra sits at the dinner table wearing a blue shirt and jeans
She took such a strong stance [against fracking waste] at the beginning
I don’t see how you can be against something then for something of this caliber.”
Styra worries about runoff water from the pits—“I don’t believe a 4-foot berm will stop that water”—and about contamination
and it looked like a small river under my culvert there,” he says
He’s also concerned about the fumes and about potential spills: “I don’t want to up sticks and go
saving yesterday to live tomorrow with the blessings from the good Lord above.”
he works shifts on an offshore drilling rig
He just doesn’t think the disposal well should be so close to people’s homes—or the nearby school
As for the scripture he wrote on the signs opposite the proposed disposal well site
“It’s a message to them and anyone else who sees it,” he says
“That everything has a moral and a story to it
Marvin Pilsner dangles his legs off the back of his flatbed
sipping cherry soda under the shade of a Chinese Pistache tree
He’s called Nordheim home for each of his 79 years
There are only a handful of houses out here off Hohn Road; it’s mostly fields
Pilsner likes to call his plot of several hundred acres “the suburbs.”
He married a girl from Cuero—the 1956 Gobbler Queen (Cuero holds an annual Turkeyfest)—and worked as a mechanic at a little shop on his property until 1984
but by 2005 he’d quit farming and leased out some of the land
but now he’s worried: More trucks have been rumbling down Hohn Road before disappearing along a dirt track that leads to the site of the proposed waste disposal pit
It’s the only thing that breaks the silence of a late-summer afternoon
To support journalism like this, donate to the Texas Observer.
Born in London, Alex Hannaford cut his teeth in journalism on the South China Morning Post in Hong Kong and newspapers on the south coast of England before joining London’s Evening Standard as a feature writer and later commissioning editor
culture and human rights issues for publications like British GQ
The Sunday Times and Sunday Telegraph magazines
All of the Texas Observer’s articles are available for free syndication for news sources under the following conditions:
You can chip in for as little as $3 a month
Get our latest in-depth reporting straight to your inbox
© 2021 The Texas Observer. All rights reserved. Site made in collaboration with CMYK
This website is using a security service to protect itself from online attacks
The action you just performed triggered the security solution
There are several actions that could trigger this block including submitting a certain word or phrase
You can email the site owner to let them know you were blocked
Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page
SAN ANTONIO - An earthquake was detected north of Nordheim
According to the United States Geological Survey
and reportedly measured a 4.0 on the Richter Scale
Two more earthquakes were detected on Tuesday
One measured in at a 2.8 and the other was a 3.0
TexNet has reportedly detected over 200 significant seismological activity within the same areas
each being measured at a 1.5 or higher on the Richter Scale
Jack Beresford is a Newsweek Senior Internet Culture & Trends Reporter
His focus is reporting on trending topics on the Internet
he covers viral stories from around the world on social media
Jack joined Newsweek in 2021 and previously worked at The Irish Post
You can get in touch with Jack by emailing j.beresford@newsweek.com
either observed and verified firsthand by the reporter
or reported and verified from knowledgeable sources
Translations may contain inaccuracies—please refer to the original content
A 100-year-old World War II veteran has opened up in a video that has touched millions about the one person from his past he would most like to see again: his mom
or Uncle Jack as he is known to his followers
but what he did have was a special bond with his grandnephew
"Uncle Jack and I became best friends when I was about 9 years old," Vonn told Newsweek
"We forged our friendship hiking and exploring the desert trails of Southern California
imparting knowledge about the wild things all around us."
Vonn began caring for Uncle Jack on a full-time basis
It was tough and soon Vonn found himself "creatively finding ways to pay the bills." That ultimately led to the creation of a GoFundMe for Uncle Jack in 2022
which spoke of Uncle Jack's desire to fight "the loneliness" of life
touched a nerve with many and soon the social media accounts Vonn had created under the handle "Ask Uncle Jack," which had been designed to support the fundraising
He now boasts an audience of some 2 million followers across social media, with clips showcasing Uncle Jack's art and poems alongside the wisdom and memories he has accumulated over his 100 years on Earth
It's there that Uncle Jack offered up arguably his most personal and powerful video to date—a clip that has garnered over 3.8 million views and counting
Vonn asks Uncle Jack: "If there is anyone that you had in your past that you could spend one day with
Uncle Jack ponders the question before responding: "It'd probably be my mama." Elaborating further
Uncle Jack explains that "she was lost because my father had died and I was trying to take care of her."
Asked what he might say to her if she walked up right now
thank you.' I'd ask her where she had been and how she is."
"She was lost when my father died." Asked what his mom might say if she saw him now
he responded that she would likely say: "Jack
So everything is OK there." Uncle Jack imagined himself replying: "Oh
"They've always gotta leave and there I am
with Uncle Jack left to look after his mom
The powerful and honest nature of his answer touched many
and feeling alone tired of the cement world and miss my mom too," one TikToker wrote
while another added: "Nobody prepares us for how lonely life can become as we get older
I'm far from 100 but I feel this." A third promised: "You'll see her again one day uncle jack."
Vonn said they tend to create their videos "in the moment," rather than starting with a clear idea in mind
He cited the emotional connection Uncle Jack makes with his followers as crucial to the clip's success
"When Uncle Jack truly takes in the moment
and decides on his 'mama,' you feel it," he said
Tears instantly appeared for me and I had to hold them back
That's my great-grandmother he's referencing
I believe his followers are right there with us."
Vonn said that Uncle Jack's reaction was just as powerful for those watching on TikTok
He said: "Seeing a 100-year-old feel this deeply about someone he's lost
I believe almost instantly brings us all to that person we miss the most
"These types of conversations bring his inner child to life
and he's no longer 100 in the year 2023; he's transformed into being a little boy holding his mama's hand
Vonn hopes people online will continue to connect with Uncle Jack's content and the fresh perspective his wise old head brings on things
"When you're scrolling between a huge variety of influencers
fighting for your attentions and all of sudden
it really makes you pause and get real for one to two minutes," Vonn said
Update 8/28/2023 9:45 ET: This article was updated to correct the spelling of Damon Vonn's name
Newsweek is committed to challenging conventional wisdom and finding connections in the search for common ground
Newsletters in your inbox See all
Thomas Von Nordheim left London for Kent and has found thriving arts and foodie scenes
Homes & Property | Where to live
renting and decorating in London from our award-winning experts
I would like to be emailed about offers, event and updates from Evening Standard. Read our privacy notice
The art of property buying often comes down to luck and good timing. And Thomas Von Nordheim’s decision to move from London to Kent could barely have been more auspiciously timed
He and his husband moved to the coast just in time to be well settled into a newly renovated flat just before the pandemic began. And they missed both the mid-pandemic race for space
and the dramatic increase in construction costs which would have impacted the work they have had done on their new home
Thomas moved to London from his native Germany to pursue a career in fashion design in the 1990s
The fundamental reason Thomas and Chris decided to get out of London was its sky high house prices
“We wanted our own space which we could do what we liked with,” explained Thomas
He was also fed up with the capital’s grime and pollution and wanted the chance to breathe in some fresh sea air
For two years before the pandemic the couple spent their free time touring locations on the Kent and south coasts, from Hastings to Margate
Eventually, since Chris, who worked for Camden Council, would have to commute to work, they picked Folkestone for its high speed rail links
Services to St Pancras International take less than an hour
“Folkestone is still a bit rough around the edges,” said Thomas
“But since we moved here it has changed quite a bit
The arts scene was here but there are more shops
a seafront promenade lined with a range of bars and restaurants
And investment is changing the face of Folkestone’s whole seafront
a contemporary apartment building overlooking Mermaid Beach
Thomas and Chris’s property of choice was a three-bedroom flat in The Grand
It cost £250,000 – about half what a similar sized home would have cost in London
During 2019 Thomas oversaw a full renovation
stripping the flat back to its bare plasterwork
and converting one of the bedrooms into a library
“It had been owned by an old lady and I don’t think it had been decorated since the 1980s,” he said
Being able to design his own home after decades of renting was a thrill
and allowed Thomas to finally use the stockpile of furniture he had been collecting for the day he had his own home
“I have been collecting pieces of furniture for years – I had things stored in garages,” he said
“If I saw something I liked for a good price I would buy it.”
The move was a slight shock to the system – Thomas was astounded to realise that provincial shops close promptly at 5.30pm – but he loves being close not only to the coast but to the Kent Downs
“We can go for a four hour walk and not see anybody,” he said
Prince Louis steals the show at VE Day parade as he keeps dad William looking sharp and mimics brother George
Prince Louis steals show with sweet antics at VE parade
VE Day 2025 fashion: best looks from the day
VE Day 2025 fashion: Princess of Wales to Lady Victoria Starmer
Ukraine 'launches stunning Kursk offensive' in major blow for Putin ahead of Victory Day celebrations
Ukraine 'launches stunning Kursk offensive' in blow for Putin
Rihanna shows off baby bump at star-studded Met Gala 2025 as singer's third pregnancy with A$AP Rocky announced
Rihanna debuts baby bump on star-studded Met Gala blue carpet
Stacey Solomon 'regrets doing reality show with Joe Swash' for tough reason
Stacey Solomon 'regrets reality show with Joe Swash' for tough reason
Homecoming Queen and King: Kyndil Villarreal and Tristan Infante
Coming Home Queen and King: Christine Wisian and Matthew Kiser
To access content, please login or purchase a subscription
*Editor's note: This story has been updated throughout.
state regulators have granted new life — if just a breath — to the tiny South Texas town’s effort to thwart an oil and gas waste site that locals argue threatens their way of life
The Texas Railroad Commission on Tuesday voted to delay final approval for a dump that would store more than a million cubic yards of waste — such as drill cuttings
fracking sand and other toxic substances — just outside the boundaries of Nordheim
which has a population of about 300 and is a mile away from a school.
The town fears that pollutants would leach into groundwater
that winds would carry volatile compounds outside of the site and that heavy truck traffic would tear up roads
In front of at least 60 residents and the entire 14-member senior class at Nordheim High School
the three-Republican panel voted to give the commission’s staff more time to consider the permit for Pyote Reclamation Systems to build the facility
The facility's 143 acres would be about half the size of the town.
the commissioners told staff to consider whether the San Antonio-based company’s proposal included enough protections for worst-case scenario rainfalls in the area
Though the commission’s staff — and officers presiding over a contested case hearing — had already recommended that the permit be approved, Grant Chambless
told the commissioners ahead of the decision: “If I had the chance to revisit the application
I would add some contingency and a safety factor.”
The vote prolonged one of the first organized protests in the heart of South Texas’ drilling country.
and we’ll worry about expeditiousness second,” said Commissioner Ryan Sitton
Commissioner Christi Craddick expressed dismay at adding more time to a process that has already stretched more than two years
saying the commission was “dragging our feet,” and that Pyote and the folks in Nordheim “deserve an answer.”
But she and Chairman David Porter eventually sided with Sitton in sending the permit back for analysis specifically on whether the permit’s current protections against a 25-year storm were enough to protect an area not far from the Gulf of Mexico
Some Nordheim residents proclaimed the decision a triumph — if only a temporary one
because it’s a little chink in the armor [of Pyote Reclamation] that it didn’t get approved," said Howard Anne Baumann
whose family owns a hayfield and rental property that would border the waste site.
but they’re going to approve it anyway,” he said
Drilling and fracking a well leaves operators liable for vast amounts of waste. They typically get rid of liquid waste by sending it away to be injected into disposal wells deep underground
But operators must send solid waste such as drill cuttings
fracking sand and other toxic substances to above-ground sites
Pyote’s facility would handle that type of waste
The site would include a mix of lined disposal pits and land treatment cells where more benign waste would be scattered and allowed to mix with soil
The company says the Nordheim site is ideal for waste disposal
It sits not far from the highway and is surrounded by drillers needing to unload waste
Pyote says its tests show that a 25-foot-thick layer of clay lies beneath the land
providing an extra level of protection for the groundwater underneath
Because the Railroad Commission’s jurisdiction is mostly limited to water impacts
the agency says it cannot factor in concerns about traffic
the Texas Commission on Environmental Quality
has sought to assure residents that Pyote wants to be a “good neighbor.”
“We certainty understand the concerns of the community
and we’ve done everything we can to respond to those concerns in different ways,” he told the commissioners Tuesday. That included adding plans for several groundwater monitors around the area
But more than 200 people — including two state lawmakers and the county judge — have sent letters to the Railroad Commission asking the oil and gas regulator to deny Pyote permits to build and operate the facility.
That includes Rep. Geanie Morrison
“Anything that happens in their community impacts everyone,” she told the commissioners
“The people who live in Nordheim do not live in the upper income bracket… If something should happen
they are not in the position to relocate.”
Morrison applauded the commissioners’ vote to delay a decision
calling the drawn-out process “very unusual” and evidence that Nordheim’s concerns were being heard
“I’m just very thankful to the Railroad Commission that they gave the time to the citizens,” she said.
no state agency is charged with considering broader land use concerns like the new trucks expected to rumble down already cracked local roads
Dewitt County Judge Daryl Fowler said Tuesday that he expects the county will shell out far more money building and maintaining roads around the site than the project would yield in tax revenue
“I have plenty of work to do in drilling and completion areas to have to worry about 100-plus trucks” Pyote would bring
Morrison acknowledged what she called “gaps” in the regulatory system for such projects
and suggested that lawmakers spend the lead up to their 2017 session examining how the Legislature could plug them.
“I think it could be something that is definitely brought up during the interim,” she said.
It’s not surprising that Morrison would stand with her constituents in opposing the dump
but her successful push last session to scale back contested-case hearings — the very mechanism Nordheim has used to protest Pyote’s application — has complicated her position.
Contested-case hearings resemble trials in which companies and their critics present evidence and testimony in front of an administrative law judge in the hopes of swaying regulators
denying or modifying a permit application.
Senate Bill 709
which she sponsored in the House at the request of a wide-range of industries, overhauled the hearings process in a variety of ways
narrowing who is considered an “affected person” who can bring a protest and arguably shifting the burden of proof from the company to the public
Morrison told the Tribune that critics of her position on the bill do not understand what it changed
“It definitely would not have affected this particular case.”
may soon have a 200-plus acre waste disposal plant as its neighbor despite the protests of the city’s mayor
The small town (one bank, one school, one cafe and a couple of shops) is located in the Eagle Ford Shale region of southern Texas
where vast oil deposits have only recently become accessible through hydraulic fracturing — a process that involves injecting a mixture of water
sand and chemicals at high pressure underground to fracture the rocks and release the oil inside
Payne learned about the waste disposal plant in the local paper
Because the facility is planned for outside city limits
industry doesn’t have to share its plans with her
Since reading about the proposal she has done all she can to learn what it will mean for her city with the assistance of Louisiana-based environmental scientist Wilma Subra
who investigates industry hot spots to help citizens make informed decisions about developments coming their way
The waste disposal site proposed by San Antonio-based Pyote Reclamation Service will be a quarter of a mile outside of Nordheim if it’s granted a permit by the Texas Railroad Commission
the regulatory agency for all things gas and oil in Texas
The facility would have eight pits up to 25 feet deep and span an area almost as big as the town itself
Pyote also has plans to install another facility 3.5 miles away
tracked down the firm’s studies for a closer look
three or four inches thick,” she told DeSmogBlog
“The first 30 to 40 pages tell you what is going on.”
Subra will share her findings at a public meeting in Nordheim on February 3
She has a good idea about which toxic materials will be dumped at the plant based on investigations she’s done into similar facilities
“The plant would have volatile [organic] compounds
including hydrogen sulfide and toxic heavy metals including arsenic
lead and chromium that all can be emitted in the air and carried off-site by the wind,” she says
“Emitted toxins from waste disposal facilities will affect an area from five to seven miles away.”
the biggest downside of the fracking boom for the town of Nordheim has been the 18-wheelers passing through
the installation of a giant waste disposal plant where fracking industry waste would be trucked in from a 100-mile radius will undoubtedly bring more unwelcome changes
Payne fears the waste at the disposal plant will contaminate the air and the water
She says the company told her the chemicals will never run off into a nearby creek that feeds into the San Antonio River
but she worries about what would happen in the case of a flash flood
She also worries that her first responders aren’t equipped to handle an industrial-scale fire
Payne says Pyote representatives assured her the plant will never catch on fire
But many of the materials Subra suspects would be dumped at the plant are flammable
“What makes them think lightning won’t hit it
Lightning hit an injection well close to the city and blew up the tank,” Payne says
Payne takes issue with the company’s use of the word “never.” To her
Payne intends to present her objections to the Texas Railroad Commission personally when it holds a hearing about the permit in the near future
She doesn’t see a need to hire a lawyer or use the industry’s technical terms
believing it is enough to cite common sense facts
“Surely if the contaminants that get on trucks within the disposal plant are toxic enough to merit cleaning before they leave the facility
the toxins are a danger to the city,” Payne says
Since the Texas Railroad Commission’s regulations do allow for some leaching of contaminants into the soil
Payne is concerned the shallow aquifer that supplies water to the area could be contaminated
DeSmogBlog asked the Texas Railroad Commission what circumstances could lead to denying a permit for a waste facility so close to a city. Spokesperson Ramona Nye sent a link to the rules they follow
When asked if the commission had turned down any proposed waste disposal sites in 2013
explaining: “This information is not readily available and would require staff research.”
“Government regulation is not a popular idea
but you have to regulate some things,” Payne says
we had one well in DeWitt County and in 2013
10,493 wells were drilled in Dewitt County.”
Payne points out that the fertilizer plant that blew up in West Texas hadn’t been inspected in years and the facility in West Virginia that contaminated the area’s water supply hadn’t been subject to inspection recently either
the worst-case scenario is a citizen getting sick from the waste facility
Does she think she can stop the Texas Railroad Commission from approving the permit for the waste disposal plant
but I am going to do what I can to protest this anyway.”
Stay up to date with DeSmog news and alerts
Even as the mood at Edmonton’s annual expo turned cautious
industry still bet on public dollars to keep its net zero dream alive
private equity firm KKR contributed to the president’s swearing-in ceremony
Despite widespread public support for clean energy and climate action
Nigel Farage’s party is running on an aggressively anti-net zero ticket
Newsletter
Website by SeriousOtters
Subscribe
Donate
at Martin - Grau Funeral Home in Waukon with Pastor Ken Kimball officiating
Burial will be held at a later date in Big Canoe Cemetery in rural Decorah
May 18 at Martin - Grau Funeral Home in Waukon
the daughter of Dervin Bernell and Phyllis Corrine (Knutson) Faldet
She was baptized at Canoe Ridge Lutheran Church in rural Decorah and confirmed at Calmar Lutheran Church in Calmar
Julie graduated from South Winneshiek High School in Calmar in 1966
she married James “Jim” Edward Nordheim at Calmar Lutheran Church
WI for a couple of years and she worked as a telephone operator
they moved to Waukon and lived on the Tom White dairy cattle farm
Julie worked at the IPLA Sale Barn in Waukon as an office manager for over 20 years
she began working in home health care for Veterans Memorial Hospital
Julie loved working with the clients and they all brought her so much joy
Her grandchildren loved visiting her at The Spectrum Thrift Store to see what treasures they could find
but still kept in touch with her special friends at Spectrum
particularly working in her garden and watching and feeding the birds
Jim and the family spent almost 40 summers at Camp Lacupolis in Wabasha
She was an avid sports fan and knew the ins and outs of all her favorite teams
She especially loved the Iowa Hawkeyes and her grandchildren’s teams; Julie was the number-one fan at all of their games
Julie also enjoyed spending time with her family and friends
which was their favorite thing to eat at her house
She looked forward to the annual Christmas shopping trip with her daughters
where they would share a Mabe’s pizza together
to keep her company in between visits from family and friends
Survivors include her children: Amanda (Jay) McGeough
Aaron Nordheim and Amy (James) Johnson; her grandchildren: Caleb Nordheim
Randy (Patty) Nordheim and Michael Nordheim; and several nieces and nephews
She was preceded in death by her father; her husband
Honorary casketbearers are Julie’s grandchildren. Online condolences may be left at www.martinfunerals.com
Jump back to navigation
Copyright 2012 | The Standard Newspaper | All Rights Reserved
4th grade Buckle Sponsor: David and Peggy Johnson; Buyers: Ray Leister- Nordheim FFA Fundraisers
Zach Leitz and Ellie Thieme – Waste Management
Not pictured: Larry Schuenemann – Speaker Sponsor
To access content, please login or purchase a subscription
NORDHEIM — Paul Baumann proudly listed his hometown’s features as he drove his Ford truck down its empty Main Street: “One grocery store
one beauty salon and one shooting club.” He pointed to the local school that houses students from kindergarten through 12th grade — which recently underwent a major remodel thanks to a $3 million bond vote
But Baumann’s tone turned somber when he addressed what he and his neighbors are fighting to keep out: a 143-acre oil and gas waste plant that developers hope to build just outside of this town of about 300
“This is going to affect the whole area with the smell,” said Baumann
who retired from the DuPont chemical company and owns ranchland
The proposed facility would border Baumann’s hayfield and a rental property
I won’t be able to rent it out,” he added
The proposed site — about half the size of Nordheim — would accept truckloads of solid waste from the surrounding Eagle Ford Shale and hold millions of gallons of toxic sludge from drilling and hydraulic fracturing
But those assurances have failed to quell one of the first organized protests in the heart of South Texas’ drilling country — a sight that could become more common as energy producers search for places to dispose of their leftovers
Signs with messages like “Don’t dump on Nordheim” have cropped up along the highway and country roads between here and Yorktown
And more than 200 people — including two state lawmakers and the county judge — have sent letters to the Railroad Commission of Texas asking the oil and gas regulator to deny Pyote permits to build and operate the facility
They have shared fears that pollutants would leach into groundwater
The Railroad Commission’s engineers have expressed support for Pyote’s application
And because the agency’s jurisdiction is mostly limited to groundwater impacts
it cannot factor in concerns about traffic
“There’s no concept of land use planning
for those types of facilities,” said Jim Bradbury
Bradbury said that while these types of facilities were necessary for shale development
there was no forum for locals to say that “they shouldn’t be right here.”
Not everyone in the area is opposed to the waste facility
a Yorktown businessman who owns the land Pyote is considering
he sent letters to local residents assuring them that the facility would be “state of the art” and “comply with the strict environmental standards established by regulatory authorities.” Calls to Dlugosch’s listed number were not answered
Drilling and fracking a well leaves operators liable for vast amounts of waste. They typically get rid of liquid waste by sending it away to be injected into disposal wells deep underground
The Railroad Commission has permitted 20 such facilities statewide
and it is considering permits for two dozen others
About 30 Nordheim residents — a 10th of the town’s residents — attended a three-day hearing last month in Austin to contest Pyote’s permits.
The hearing resembled a trial in which companies and their critics presented evidence and testimony in front of an administrative law judge
who later issues a nonbinding decision that might sway regulators’ opinions
the judge’s decision is likely months away — and the Railroad Commission will make its decision after that
“We moved here to live and to be old,” Barbara Fulbright
who lives less than two miles from the proposed site
No one wants to live on a road with oil and gas waste.”
assured those at the hearing that “Pyote is certainly not going to be doing anything on the cheap,” and that “they want to be good neighbors.”
they have the right to use their property in any manner that is legal,” he said
providing an extra layer of protection for the groundwater underneath
so it did not have details on the proposal
But the facility could be subject to air nuisance laws once it is up and running
Though many people following the fight expect the Railroad Commission to grant Pyote permission to open the facility
the commission denied Pyote’s initial application
Its letter to the company said the plant “may cause or allow pollution to surface or subsurface waters,” and identified 38 areas of concern
including those related to the strength of the site’s proposed waste pit liners
its proximity to water wells and its ability to capture runoff after rains
After Pyote responded to those points and made changes to its application
the commission’s staff recommended the permit
Sitting in the living room of her ranch house just down a dirt road from the proposed waste site
Lynn Janssen said such a decision would only complicate life for her fellow ranchers
“It’s a struggle because you gamble against the weather,” she said