I am student of network security, and very unfamiliar with the domain of signal processing. Do pardon my naivety at asking very basic stuff.

My work deals with detecting malicious activity in network logs -say, differentiating between botnet traffic and benign Internet traffic. I came across this interesting article titled Detecting and identifying malware: a new signal processing goal. Since then, I have been exploring the world of FFTs, DTFTs, Wavelet transforms, cepstrum and the ilk. But I have not been able to make much progress with the application of these principles.

I wish to know from dsp experts that what kind of time-series transforms can be useful for my task.

To keep things clear, I will provide a brief description of data I deal with: As I said, my task is of separating botnet traffic from Internet traffic. Lets define Internet traffic as the traffic seen from the systems of regular users of the Internet like you and me, who browse the web, watch a few videos, check mail, maybe download a few movies, and also exhibit periods of inactivity say while they sleep, eat, etc.

And lets define botnet traffic as the traffic of a covert malware/virus (a 'bot') on your machine calling home to its master or contacting fellow 'bots' (just to maintain connectivity), and also the commands sent by the master to its bots or those exchanged by bots amongst themselves. This traffic does typically have patterns and periodicity.

If I am right, I am dealing with signals of botnet traffic mixed with Internet traffic (which does not have any specific pattern?) What kind of time-series transforms can be useful here? DFTs? FFTs? Cepstral analysis {or is that being too creative? :)}


closed as too broad by Jason R, Paul R, Peter K. Mar 21 '14 at 0:02

Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • 2
    $\begingroup$ In general, traffic analysis by simple packet count/frequency is hard because you are not using relevant information revealed by packet inspection and analysis of hosts involved. Please remember that an adversary may choose to masquerade his traffic easily. $\endgroup$ – Deer Hunter Mar 4 '14 at 9:04
  • $\begingroup$ @DeerHunter I tried to put things in a very simple way since I do not know if dsp experts reading this question will be security experts too. I am taking care of looking at some other features- like payload variation, inter-arrival time of packets- separately. If your intention was of DPI- no, I am not doing that because encryption anyways renders it useless. $\endgroup$ – pnp Mar 4 '14 at 10:34
  • $\begingroup$ btw, DPI means Deep Packet Inspection $\endgroup$ – pnp Mar 4 '14 at 10:35
  • 1
    $\begingroup$ @pnp: I think this question is too broad. Since most signal processing people aren't also computer security people, it's hard to say what techniques would be helpful for your application. It all comes down to what the signals look like and what any interference or noise looks like. Also, a lot of signal processing is based upon the properties of linear time-invariant systems; it isn't clear that such a concept would have an analog in your domain. $\endgroup$ – Jason R Mar 4 '14 at 12:18
  • $\begingroup$ @JasonR I get your point. But honestly, I was just looking for some creative inputs from dsp people who have some understanding of networks and security. $\endgroup$ – pnp Mar 4 '14 at 12:29

The point seems to be moot. An attacker could hide in noise. Any meaningful interpretation of the question appears to aim for temporal regularity of attacks. Maybe an attack is temporally regular, but so is users' behavior. So, there might be lots of applications for transforms in malware detection, but none of them appears to promise any insight on how to recognize or intercept it.

From a computer science standpoint i would say that ,,recognizing malware'' is akin to the problem of recognizing two functions as equal, which in general is a proof, which is not computable in any ordinary sense. It would probably amount to proving P=NP. Any ,,heuristic'' recognition of malware can be evaded that way. The only system capable of actually and reliably recognizing malware appears to be a conscious human being.

  • $\begingroup$ Thanks for your inputs. As a student of network security, allow me to deal with the headache of finding the attacker trying to hide behind noise. As a dsp expert, can you give pointers to how to use these techniques for my purpose? Also, your last statement is in direct contradiction to all research efforts towards automated detection of malware and network threats. A LOT of research has been done on that... $\endgroup$ – pnp Mar 5 '14 at 5:51
  • $\begingroup$ Research does not imply success, sadly. I am not aware of any way of using time series to recognize malware that does not appear to be both obvious and at the same time pointless. If you find one, be sure to answer. Please ask another computer scientist about the CS stuff if you feel I am wrong. $\endgroup$ – user7358 Mar 5 '14 at 8:06
  • 1
    $\begingroup$ Agreed... research does not imply success. But we researchers can't stop doing our job for that ;). Btw, you may want to look at this research paper I found: Online Botnet Detection Based on Incremental Discrete Fourier Transform $\endgroup$ – pnp Mar 5 '14 at 9:33

Not the answer you're looking for? Browse other questions tagged or ask your own question.