How to start

There are many ways to approach this. I will point out an easy way, whereas one has to see, that this is a computer based one, which is not the only way to go, but the one of the most efficient. You can start to make music with points 1, 2 and 3 the rest is optional. I list here a logic, and in my eyes, smart order of buying equipment, having a low budget in mind.

Computer


The first thing you need: a strong personal computer.

What is strong? We really have the luxury now that we can say, that almost every computer you can buy today is fast enough. Not counting useless netbooks though. But faster is (of course) always better or lets say at least more convinient. Even what you can achieve with an ipad is impressive.

Here some more or less timeless smart things to consider when buying or building a computer especially in the windows world:

If you build one yourself never spend more than about 350 Euro for the CPU. Those which cost more than that are new and overpriced. Buy at least 8GB (2014) of memory and realize that you HAVE to use 64bit OS to make use of that, choose silent harddrives. Calculate the best "price per gigabyte" ratio (divide the prize by the gigabytes) and go for the hardrive-size with the lowest price per gigabyte.

If you go for a motherboard choose known brands and intel chipsets, google for reviews for the board of your choice and look what they say about stability (I prefer Intel and Asus mainboards). Furthermore go for a Graficcard with two outputs and passive cooling, don't pay more than 120 Euro except if you want to play games. At any time since 2006 you can build or buy a good music computer for 600-1000 Euro. Spending more than that is often not efficient since you pay too much for needless "new" technology.

If you go for a laptop go for known brands. Dell is cheap and mostly good, Sony is very good but expensive, Asus, Toshiba, IBM Lenovo are quality too. Read reviews, all sometimes release crap. For sure buy a silent notebook, make sure you test it when CPU is in full use, because notebooks only reveal their noise when they get hot. 
If you want a mac you can make do with a MacBook Air (choose one of the quicker models) or also with a mac mini. I use an iMac and it's working very well. Basically if you get a mac i doesn't really matter what you buy. It all works well. You can go back to 2012 Models with no problems in the second hand market.