Is Abraham's covenant through Isaac or Ishmael?

According to the biblical record, God's eternal covenant of promise and spiritual inheritance is explicitly established through Isaac, not Ishmael, even though God promised to bless Ishmael physically as a nation.