LaTeX is a document preparation system that was originally developed for typesetting mathematics articles; however, it has evolved over the years into a tool for people who focus on producing publication quality articles with scalable graphics.
LaTeX is the go-to typesetting program for mathematicians and others in STEM due to its relatively level learning curve and its low cost (free). Additionally, there are a number optional software packages and extensions which can simplify difficult and repetitive tasks within LaTeX. Using these tools, LaTeX users can produce astounding results in terms of looks and formatting.
BibTeX is an extension of LaTeX used for generating and formatting bibliographies and reference pages. More often than not, citation management software and journals are able to citations in a BibTeX-friendly format. These citations are placed into a separate citation file (.bib), which is then referred to by your LaTeX document (a .tex file).
One of the more convenient features of BibTex is that there are packages that will format your references after you place your citations in the document, so the numbering will update after you update your citations, without any hassle. Some of these packages include the following:
Many journals, particularly in STEM subject areas, will automatically convert submitted manuscripts into *TeX documents as part of their production process. If you can submit your article to them in this format (some journals even encourage this, providing pre-made templates), it saves them time and hassle.