My Math Forum  

Go Back   My Math Forum > College Math Forum > Real Analysis

Real Analysis Real Analysis Math Forum


Reply
 
LinkBack Thread Tools Display Modes
August 19th, 2012, 03:07 AM   #1
Newbie
 
Joined: Aug 2012

Posts: 1
Thanks: 0

Interpolation of larga data sets

This is my first post so I wish to say hello to everyone Hello!

Here is my problem. I have fairly large data sets of around 100-200k of natural numbers for which I wish to find a function, relation, series or a sequence (I don't really care). For the purposes of a discussion, let's say it is a series. The only thing this series has to be able to do is map element "k" (0...k...n) to a value from the dataset. I do not care what happens to the series after the final value or how it behaves before first element. Another issue is that errors are only acceptable at decimal point (ie. if index 5 gives value 110, then it is OK if it shows anything between 9.5 and 10.49 but nothing else). All values have to be mapped and no additional values can show up. Can you help me?

I made some research and I think Gauss's Interpolation Formula might be what I am looking for, but I am not sure. Also, I do know neural network could be trained to fit the data, but it is too CPU expensive since I would have to train many networks a day (which is why I am looking for alternative methods).
godric is offline  
 
August 31st, 2012, 06:53 AM   #2
Senior Member
 
Joined: Aug 2012

Posts: 229
Thanks: 3

Re: Interpolation of larga data sets

Hey godric.

You can interpolate between the points quite easily but the function you create will be very complicated and probably just as complicated as the information embedded in the actual points themselves.

Typically in modern mathematics, what happens is that you use what are known as projection based techniques.

Projection based techniques can take a signal of some sort (like your points) as well as a model of a function and it can project those points to the basis vectors corresponding to the function and then recombine the results to get an approximated version of the signal if the basis you are using is not of infinite-rank (just think of it as an approximation of the signal and not the absolute representation for simplicity).

So if you want to use a particular model like a bunch of wave functions, some nth degree polynomial or some other function, you can construct a basis on an interval, project your signal to that basis and then re-construct an approximation of your signal with respect to that basis by using the theory of Hilbert-spaces and the infinite-dimensional vector space theory concerning the results of projections onto L^(R).

If you are not going to use something like a neural network, you will need to decide the basis yourself: I would suggest that you choose a fourier series over the interval and narrow the band (i.e. keep as many lower frequencies as required).

I can run through this with you if you don't have much math experience, but the idea is that this is basically a way of reconstructing your signal using the standard signal processing idea of treating the input as a bunch of sines and cosines and extracting the co-effecients for each set of individual frequencies.
chiro is offline  
Reply

  My Math Forum > College Math Forum > Real Analysis

Tags
data, interpolation, larga, sets



Thread Tools
Display Modes


Similar Threads
Thread Thread Starter Forum Replies Last Post
Two data sets fitting Simon39 Abstract Algebra 4 November 29th, 2013 10:41 PM
Combining 2 data sets Shakarii Advanced Statistics 1 August 10th, 2012 03:32 AM
Math - Data Management problem involving sets and subsets. police1.police2 Algebra 1 December 16th, 2011 01:57 PM
Help on comparing differing sets of data carter22 Algebra 2 March 1st, 2010 06:49 PM
Comparing multiple data sets with different n values cboldon Advanced Statistics 2 April 7th, 2008 04:01 PM





Copyright © 2019 My Math Forum. All rights reserved.