Your level of concern on brick cracks in your wall is directly associated with the type, location, and nature of the crack.
Keep in mind brick is strong in compression (forces sitting on top of it) but brittle. A small hairline crack in a single brick is of little concern. A large, jagged crack through several bricks, wide enough to put your finger into should be of considerable concern and you should have a competent professional look into what may be causing it. Most of the time, the brick you see on your wall is not hold up your house.
These examples are the extreme ends of the spectrum regarding brick cracks. So, let us consider some typical examples of brick cracks:
On a brick veneer home, metal lintels are commonly installed above window and door openings to support the brick above the opening. These metal lintels often rust over time and the rust can build up to the point it pushes on the brick enough to crack the brick and/or mortar. Some homes have wood timber lintels, which may shrink or warp over time. Warping can exert forces on the brick and shrinkage can reduce support for the brick sitting on the wood. Both conditions can result in cracks. Cracks of this nature can typically be adequately addressed with sealant or grout. Long lintels over garage door openings commonly result in brick cracks at each end, indicating the lintel is not providing adequate support. However, other framing issues can also contribute to these same cracks and knowing the difference requires a trained professional to diagnose the problem.
Long zig-zag cracks running from the foundation to the roof may be indicators of foundation issues. Long horizontal brick cracks are often the result of framing deficiencies in the structure. Both types of cracks should be referred to a professional for analysis.
Bricks expand and contract with changes in the weather or temperature. On a long continuous wall, the expansion of the brick wall (yes, it expands when the sun shines on it) can exert enough force on other bricks to crack them, particularly around window openings or at corners of the wall. This expansion can even crack the corners off the concrete foundation.
A little time spent with a competent professional can provide an answer regarding how much you should be concerned about brick cracks and provide peace of mind.