Yes, definitely. I think where folks get thrown is that being in covenant with God is primarily dipleuric, i.e., it is two way, and one can violate the terms and be punished as a covenant breaker. And there is a monopleuric sense of the covenant, only for the elect, wherein Christ keeps them. Like what I quoted from Olevianus in another post:

Let us then see what the kingdom of Christ is, which begins in the faithful in this world and is also called, with the same meaning, "the kingdom of God" and "the kingdom of heaven" (Matt. 3:2; Luke 4:43; 7:28). The kingdom of Christ in this world is the administration of salvation by which Christ the King Himself outwardly, through the gospel and baptism, gathers to Himself and calls to salvation a people or visible church (in which many hypocrites are mixed). To those in this congregation who have always been His elect, He Himself administers and bestows that salvation to which He calls them. He makes the outward call efficacious, granting them the repentance and faith by which they respond to the One calling them. Those He calls in this way He also justifies, not imputing their sins to them. And those He justifies He also glorifies, purging them daily more and more of their sins, and training, forming, and perfecting them in all godliness, righteousness, and eternal life so that the glory of Christ the King may shine in them. To that end He uses the public, domestic, and private dispensation of His Word and sacraments by suitable ministers, as well as the diligent administration of His discipline, as it relates not only to repentance and ceremonies but also to one's whole life.

Christ grants the substance of the covenant, Himself with all His saving power, only to the elect. Non elect only participate in the substance of the covenant, that is, the outward means that Christ uses to save His elect. But all in the visible church are part of the Kingdom of Christ by covenant in this life.