It's his loss
In the question: He is the only one who loses because of his refusal to accept the offer made to him.
(Modern Hebrew)
Translation
Expression (Hebrew)
הַהֶפְסֵד כֻּלּוֹ שֶׁלּוֹ
Definition (Hebrew)
בְּהַשְׁאָלָה: הוּא הַיָּחִיד שֶׁמַּפְסִיד בִּגְלַל סֵרוּבוֹ לַהַצָּעָה שֶׁהֻצְּעָה לוֹ.
(הָעִבְרִית הַחֲדָשָׁה)