Building retrofit is considered as a vital step to achieve energy and climate goals in both Europe and Sweden. Nevertheless, retrofitting solutions based merely on reducing operational energy use can increase embodied energy use, mainly due to altering the existing trade-off between the two. Considering this trade-off is vitally important, especially for retrofitting buildings located in cold climate regions, as reduction of operational energy use to meet standards of energy-efficient buildings may require a deep retrofitting that can considerably increase the embodied energy and thus be unfavorable from a Life Cycle Energy (LCE) perspective. This article presents a case study in which multi-objective optimization was used to explore the impact of a wide range of retrofitting measures on the aforementioned trade-off for a building in Sweden located in a subarctic climatic zone. The studied building was a typical 1980s multi-family residence. The goal was to explore and compare the optimal retrofitting solution(s) for the building, aiming to achieve Swedish energy-efficient building standards (i.e. new-build and near-zero energy standards). The results of the optimization indicated that (1) use of additional insulation in walls and roof, (2) replacement of existing windows with more energy-efficient ones, and (3) change of traditional mechanical extract ventilation to heat recovery ventilation are the primary and optimal retrofitting measures to fulfill the new-build Swedish energy standard and achieve highest LCE savings. However, to fulfill more far-reaching operational energy savings, application of additional retrofitting measures was required, increasing the embodied energy use considerably and resulting in lower LCE savings compared to the optimal retrofitting solution that only reached the Swedish new-build energy standard. The LCE difference between the optimal retrofitting solutions that fulfilled the new-build standard and the strictest near-zero (passive house) standard was 1862 GJ, which is equivalent to almost four years of operational energy use for the original building. This indicates that there is a limit to the reduction of operational energy use when retrofitting existing buildings, beyond which additional reductions can considerably increase the embodied energy and thus be unfavorable in terms of LCE use.