On PCI analogue cards, this happens if you do not have answer supervision. Answer supervision requires support from the PSTN operator and enabling in Asterisk. I think the only analogue answer supervision protocol that chan_dahdi supports is line reversal. If you enable this and the PSTN operator doesn’t use it, calls will appear never to be answered.
If this is ISDN, you should get answer supervision at the right time. If you don’t, it will be a network operator issue.