Actually traveling in Bangkok is very cheap comparing to many countries around the world. In case that you are a budget traveler and want to save your money as much as you can, here is my recommendation.
1. Use public transportation
Even if taxi fare in Bangkok is cheap, it is much cheaper to travel around the city with public transportation. BTS Skytrain and MRT Subway currently cover many areas around the town. They are the best transportation modes for wandering around Bangkok. You would not only save money, but also your time.
2. Stay at the hostel
Hostel is of course cheaper than hotel. My recommendation is try to stay at the one that is located near either BTS or MRT stations. If you cannot find the cheap hostel that is located in the city center areas such as Siam, Asok, or Nana, I recommend choose the hostel that is located a little far from the center, but still close to the BTS or MRT. It should be cheaper.
3. Eat at street food stalls or food court
Eating at street food stalls in Bangkok is a lot cheap than eating in restaurants. You can find food stalls all over the town. They are not only cheap, but also delicious. Some foods such as Som Tum (Thai green papaya salad) are better to eat at street food stalls.
Food court could be another good option especially for people who do not know Thai food and Thai language. Food stalls in food court usually provide menu in both English and Thai. Food court is usually located in the mall. But, eating at food court is not always cheap. The cheapest and my favorite one is at terminal21 shopping mall.
4. Instead of luxury malls, shop at markets
There are many markets around the town. If you want to buy cheap stuffs, go to the markets. Most markets accept bargaining, and if you buy more, you are likely to get discount. The most popular and biggest market in Bangkok is Chatuchak weekend market, the market that you can find almost everything you want.
Besides, there are many free (or at least cheap) things to do in Bangkok. Check it out and choose the one that fits you. Hope the information helps!