Neural networks for optimal approximation of smooth and analytic functions; Neural Computation, {\bf 8} (1996), 164-177.

We prove that neural networks with a single hidden layer are capable of providing an optimal order of approximation for functions assumed to possess a given number of derivatives, if the activation function evaluated by each principal element satisfies certain technical conditions. Under these conditions, it is also possible to construct networks that provide a geometric order of approximation for analytic target functions. The permissible activation functions include the squashing function $(1+e^{-x})^{-1}$ as well as a variety of radial basis functions. Our proofs are constructive. The weights and thresholds of our networks are chosen independently of the target function; we give explicit formulas for the coefficients as simple, continuous,linear functionals of the target function.

Bounded quasi-interpolatory polynomial operators; Journal of Approximation Theory, {\bf 96} (1999), 67--85.  (With J. Prestin).

We construct bounded polynomial operators, similar to the classical de la Valle\'e Poussin operators in Fourier series, which preserve polynomials of a certain degree, but are defined in terms of the values of the function rather than its Fourier coefficients.

We investigate the relationships between the Marcinkiewicz-Zygmund-type inequalities and certain shifted average operators. Applications to the mean boundedness of a quasi-interpolatory operator in the case of trigonometric polynomials, Jacobi polynomials, and Freud polynomials are presented.

We propose a class of algebraic polynomial frames, which are computationally easier to implement than polynomial bases. We also discuss the weighted $L^p$- stability of our frames for $1\le p \le \infty$. Our analysis is based on orthogonal polynomials with respect to the weight in question, but the frame bounds are independent of the system of orthogonal polynomials used. In spite of the fact that algebraic polynomials are inherently nonlocal, our frames provide good localization properites. In particular, they can be used to detect discontinuities in derivatives of all orders of a function. We describe asymptotic expressions for the frame coefficients in the vicinity of a discontinuity.

On a sequence of fast decreasing polynomial operators; in ``Applications and Computation of Orthogonal Polynomials''
(Eds. W. Gautschi, G.H. Golub, G. Opfer)    Internat. Ser. Numer. Math. Vol. 131, Birkhäuser, Basel, 1999, 165-178. (With J. Prestin).
 

Let $f$ be a piecewise analytic function on the unit interval (respectively, the unit circle of the complex plane). Starting from the Chebyshev (respectively, Fourier) coefficients of $f$, we construct a sequence of fast decreasing polynomials (respectively, trigonometric polynomials)which ``detect'' the points where $f$ fails to be analytic, provided $f$ is not infinitely differentiable at these points.

On the detection of singularities of a periodic function; Advances in Computational Mathematics, {\bf 12} (2000), 95--131 (With J. Prestin).
 

We discuss the problem of detecting the location of discontinuities of derivatives of a periodic function, given either finitely many Fourier coefficients of the function, or the samples of the function at uniform or scattered data points. Using the general theory, we develop a class of trigonometric polynomial frames suitable for this purpose. Our methods also help us to analyze the capabilities of periodic spline wavelets, trigonometric polynomial wavelets, and some of the classical summability methods in the theory of Fourier series.

For the unit sphere embedded in a Euclidean space, we obtain quadrature formulas that are exact for spherical harmonics of a fixed

order, have nonnegative weights, and are based on function values at

scattered points (sites). The number of scattered sites required is

comparable to the dimension of the space for which the quadrature

formula is required to be exact. As a part of the proof, we derive

$L^1$-Marcinkiewicz-Zygmund inequalites for scattered sites on the

unit sphere.

A zonal function (ZF) network is a function of the form

$\x\mapsto\sum_{k=1}^n c_k\phi(\x\cdot \y_k)$, where $\x$ and the

$\y_k$'s are on the on the unit sphere in $q+1$ dimensional Euclidean

space, and where the $\y_k$'s are scattered points. In this paper, we

study the degree of approximation by ZF networks. In particular, we

compare this degree of approximation with that obtained with the

classical spherical harmonics. In many cases of interest, this is the

best possible for a given amount of information regarding the target

function. We also discuss the construction of ZF networks using

scattered data. Our networks require no training in the traditional

sense, and provide theoretically predictable rates of approximation.

On a build-up polynomial frame for the detection of singularities; in ``Self-Similar Systems'' (V. B. Priezzhev and V. P. Spiridonov Eds.), Joint Institute for Nuclear Research, Dubna, Russia, 1999, pp. 98--109.   (With J. Prestin).
 

Let $f :[-1,1]\to\RR$, $x_0\in (-1,1)$, $r\ge 0$ be an integer. The point $x_0$ is called a singularity of $f$ of order $r$ if the derivative $\derf{f}{r}$ has a jump discontinuity at $x_0$, but is continuous at every other point of some neighborhood of $x_0$. In this paper, we propose a sequence of polynomial operators $\{\tau_j\}$ with the following properties. Each $\tau_j$ is computed using the values $f(\cos (k\pi/2^j))$, $k=1,\cdots,2^j-1$, and the quantity $\tau_j(f,x)$ is ``large'' near a singularity, and ``small'' away from it. Precise quantitative estimates are given.

 

 
 
 
 
 
 
 

Quasi-interpolation in shift invariant spaces; J. Math. Anal. Appl. {\bf 251} (2000), 356--363. (With F. J. Narcowich and J. D. Ward).
 
 

Let $s\ge 1$ be an integer, $\phi :\RR^s\to\RR$ be a compactly supported function, and $S(\phi)$ denote the linear span of $\{\phi(\cdot-\k)\ :\ \k\in\ZZ^s\}$. We consider the problem of approximating a continuous function $f :\RR^s\to\RR$ on compact subsets of $\RR^s$ from the classes $S(\phi(h\cdot))$, $h>0$, based on samples of the function at scattered sites in $\RR^s$. We demonstrate how classical polynomial inequalities lead to the construction of local, quasi-interpolatory operators for this purpose.

Polynomial frames on the sphere; Accepted for publication in Advances in Computational Mathematics (With F. J. Narcowich, J. Prestin and J. D. Ward).
 

We introduce a class of polynomial frames suitable for analyzing data on the
surface of the unit sphere of a Euclidean space. Our frames consist of
polynomials,  but are well localized, and are stable with respect to all the
$L^p$ norms. The frames belonging to higher and higher scale wavelet spaces have
more and more vanishing moments

Approximation theory and neural networks; accepted for publication in  "Wavelet Analysis and Applications, Proceedings of the international workshop in Delhi, 1999" (P. K. Jain, M. Krishnan, H. N. Mhaskar J. Prestin, and D. Singh Eds.), Narosa Publishing, New Delhi, India
 

 
In this tutorial, we discuss the problem of approximation of functions by neural networks and radial basis function (RBF) networks. The first two lectures are introductory, explaining the direct and coverse theorems in classical trigonometric approximation. In the third lecture, we describe the ideas behind our work with Narcowich and Ward regarding the construction of quasi-interpolatory trigonometric polynomial operators using scattered data on a torous. The fourth lecture is devoted to our work with Micchelli on the construction of RBF networks using scattered data, but without involving any training mechanism. In the fifth lecture, we describe our results regarding the close connection between polynomial approximation and neural network approximation. Further applications to system identification (work with Hahm) and approximation on the sphere (with Narcowich and Ward)are also discussed.
 

Neural network frames on the sphere; Accepted for publication in the Proceedings of the International Symposium on Signal Processing and Neural Networks, Sydney, Australia, December, 2000. (With F. J. Narcowich and J. D. Ward).
 

We construct a multiresolution analysis of the standard Hilbert space on a Euclidean sphere, which can be implemented directly by neural networks. The neural networks may utilize any sufficiently smooth function as an activation function, and their size can be determined in advance. We define frame operators that can analyze data selected at {\it scattered\/} sites, rather than any particular set of points. The number of vanishing moments increases with the order of the frames. In particular, the frames can detect discontinuities in arbitrarily high order derivatives. The neural networks do not require any training in the traditional sense.
 

Approximation with Interpolatory Constraints; Accepted for publication in Proc. Amer. Math. Soc. (With F. J. Narcowich, N. Sivakumar, and J. D. Ward).
 

Given  a triangular array of points on $[-1,1]$ satisfying certain minimal separation conditions, a classical theorem of Szabados asserts the existence of polynomial operators that provide interpolation at these points as well as a near-optimal degree of approximation for arbitrary continuous functions on the interval. This paper provides a simple, functional-analytic proof of this fact. This abstract technique also leads to similar results in general situations where an analogue of the classical Jackson-type theorem holds.  In particular, it allows one to obtain simultaneous interpolation and a near-optimal degree of approximation by neural networks on a cube, radial-basis functions on a torus, and Gaussian networks on Euclidean space. These ideas are illustrated by a
discussion of simultaneous approximation and interpolation by polynomials and also by zonal-function networks on the unit sphere in Euclidean space.
 

On the representation of band-limited signals using finitely many bits; Submitted for publication.
 

In this paper, we consider the question of representing an entire function of finite order and type  in terms of finitely many bits, and reconstructing the function from these. Instead of making any further assumptions about the function, we measure the error in reconstruction in a suitably weighted $L^p$ norm. The optimal number of bits in order to obtain a given accuracy is given by the Kolmogorov entropy. We determine this entropy in the case of certain compact subsets of these weighted $L^p$ spaces, and obtain constructive algorithms to determine the asymptotically optimal bit representation from finitely many samples of the function. Our theory includes both equidistant and non-uniform sampling. The reconstructions are polynomials, having several other optimality properties.

Home

Research

Download