In response to Kate Gosselin‘s recent tell-all, father-of-eight Jon Gosselin says he’s “very upset.”

While posing with seven of their Children on PEOPLE‘s latest cover, Kate, 41, gave details on their past marriage and updates on their eight children.

Kate and Jon’s 15-year-old twins Cara and Mady claim their father is not involved in their lives.

“He makes it seem like we’re being kept from him, which is insane,” Mady said of her dad. “He should maybe spend some time thinking about why we don’t want to see him, and maybe realize that if he ever does want a relationship with us, talking about us on TV is not the way to make that happen.”

“He doesn’t even know us,” Mady added. “How can he dare to talk about us?”

In response, the father-of-eight fires back.

“I love my children very much. I’ve always been there for them and I always will be,” Jon, 39, told Entertainment Tonight. “It’s shocking.”

In addition, Kate revealed that their 12-year-old son Collin has “special needs” and is in a program away from home.

“This has been a struggle we’ve had for a very long time, and it’s one I’ve dealt with on my own,” Kate said. “I’ve felt very alone in this. By the same token, it’s not something that has only impacted me or him – our entire family has been impacted.”

Jon admits he hasn’t seen Collin in two years.

“It felt like she was segregating me from him,” Jon said of his ex, adding that he hasn’t seen all eight of his kids together in “three to four years.” Jon added, “I haven’t seen him for a really long time.”

He continued: “Everybody used to come to my house. It was great, and then Madelyn and Cara just stopped coming, around 12-13 [years old]. I didn’t question it. I was just like, ‘Hey, do what you want. Be with your friends,’ and then it just became, ‘They’re not coming.’”

Jon and Kate are also parents to Collin’s sextuplet sibs Alexis, Leah, Hannah, Joel and Aaden.