I'm trying to get into academic research at the intersection of vision and graphics, and was wondering how researchers in the field went about this:

  1. Was it useful to do a lot of fundamental math courses (Linear Algebra, Multivariable Calculus, Probability, Statistics) prior to reading papers, or did you find it more efficient to learn the math on the go?

  2. How did you get started on reading and implementing papers? What helped and what didn't? Looking back on your experience, what did you wish you were more prepared with?

  • 2
    $\begingroup$ Pick a problem. Try to solve it yourself with what you know now and the tools you have. You'll succeed to some extent, if not just to hypothesize various solutions, even if they are beyond your ability. Perhaps you succeed completely. Then go search for others' solutions, be it in academic papers or specialized websites. Compare your solution to theirs. There is likely some overlap. Your efforts should give you perspective and understanding of their solution. You are likely to learn new math and learn it quicker with an application in mind. Or a different way of doing what you did. $\endgroup$ May 27, 2020 at 3:02
  • $\begingroup$ That's some sound advice, thank you! :) When you try to solve the problem though, how do you avoid going down the rabbit hole? Let's say that when you look at others' solutions, you come across techniques that you haven't seen before, and while researching more, those techniques rely on other techniques which you don't know about, and ad infinitum. Before you know it, you're bogged down in material that you haven't ever seen before, to even understand one solution. How do you get around this problem? $\endgroup$ May 27, 2020 at 12:58
  • $\begingroup$ Phaedrus says: "Start with a single brick." In other words, pick a simpler problem. Your intersection is too broadly defined for me, so no specific advice, maybe start by writing a simple video game, then add effects. The point about reinventing wheels is that when you figure something out yourself, it is still an "aha moment" for you, and you understand it much better. Furthermore, someone else's take on that type of wheel is more meaningful. Sometimes some one happens invent a new wheel. It wouldn't have happened if they weren't trying. $\endgroup$ May 27, 2020 at 13:23

1 Answer 1


People working in computer vision/graphics may have very different backgrounds: computer programming, high-performance computing, signal processing, image analysis, optimization, machine learning, etc.

Novel contributions can be wide: from a very fast GPU/CPU algorithm to render textures (I had a reference in mind which I can't recall now) to semantic segmentation. Origins of ideas may range from huge expertise, trials on billion of images to profound thoughts, through hardware algorithmic wisdom. Your question is far-reaching, and can dig into the depth of the psychology of scientific invention, well-described by some authors in mathematics and physics (I have Raymond Poincaré and Jacques Hadamard in mind). Here are a couple of thoughts. The ability to (non exclusive):

  • be and remain curious: listen a lot, don't pretend too much, don't trust doxa (everybody knows - even yourself - that..., nobody does this), look at novelties in other fields
  • test and probe:hardware, data, hypothesis, models... to the limit (add too much noise, down-sample a lot, reduce learning set, cut the power),
  • explore unconventional ideas: transfer knowledge from one domain to the other, ask "what if I do..." (I got from Isaac Asimov that most of science is not like Archimedes' "Eureka (I found it)",but often "Ooh, that funny"), learn XXX that strange theory, go to seminars you are not truly interesting in, or above your level
  • have people you trust to talk to: especially those who disagree, or dare to say "you're going nowhere", hire good colleagues, better students
  • take notes, talk to yourself, don't be discouraged: nice works can take decades
  • teach: basic student questions can drive you nuts and shake your confidence
  • corollary: have some solid knowledge somewhere (law of excessive learning: you can teach at one level only if you know the level above)
  • breathe and sleep: get away from the machine, take some fresh air, wander in woods
  • be patient, but remain aware of your progress: one gain confiance progressively. The first "original" ideas I got were already discovered 30 years before. I'm grateful now went I read a 1-year paper that implements what I am thinking of.
  • have principles: it is OK to think "This should work!" even if the evidence is shallow. Think about the "AI winter".

Those can be good incentives. On my side, I am lucky to (think that I) have a relatively broad "generic" scientific background (to connect far-fetched domains) and one or two technical skills that I dug deep into. Thus, I am not afraid of switch domains. One main risk I see is siloing: when collaborating with other, it is tempting to (and thus, try to avoid):

  • remain in our confidence zone,
  • trust others in their domain (as we would like to be treated, with respect).

One of my few scientific prides (not deep, but quite intriguing) was to work with real-time computing people (my programming skills are very limited), and discover that some "dogma" in the domain was false. My contribution was initially very small, on standard extrapolation, which I though I mastered. Some colleagues trusted me and implemented the stuff. But one colleague who did not understand the derivation asked me again, and again, and again, to understand every step that I thought was "evident". So I reduced the formulas down to the bone. With a couple of image compression tricks, we ended up with a scheme that shook the initial dogma in cyber-physical simulation. Not theoretically, but enough to feel we had "something new".

  • 1
    $\begingroup$ Thank you so much for sharing your experience and wisdom! This has been immensely helpful for someone like me, who's just starting out in research. :) $\endgroup$ May 27, 2020 at 6:24
  • $\begingroup$ I hope I did not convey too much arrogance. I am not a top-notch researcher, more an engineer with scientific curiosity (and I enjoy the tiny domain where I work) $\endgroup$ May 27, 2020 at 9:10
  • 1
    $\begingroup$ Oh, definitely not! You are far more experienced than me, and everything what you wrote is the true essence of scientific research. Engineer/researcher are just terms. It's the work we do and the spirit with which we do them that counts. :) $\endgroup$ May 27, 2020 at 12:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.