IAX2 ACL Weirdness

We have two Asterisk 11.11.0 servers connected to each other via an IAX2 connection:

Server 1:
[server2]
host=10.50.0.5
secret=xxxxxxxxxx
type=friend
disallow=all
allow=g729
canreinvite=no
qualify=yes
dtmfmode=rfc2833
context=from-server-2
host=dynamic
callerid=asreceived

Server 2:
[server1]
host=10.50.0.4
secret=xxxxxxxxxx
type=friend
disallow=all
allow=g729
canreinvite=no
qualify=yes
dtmfmode=rfc2833
context=from-server-1
callerid=asreceived

We keep on receiving a bunch of the following notices every view seconds:

[2014-08-20 13:05:49] NOTICE[17129]: acl.c:748 ast_apply_acl: IAX2 user ACL: Rejecting ‘10.50.0.5’ due to a failure to pass ACL ‘(BASELINE)’
[2014-08-20 13:05:49] NOTICE[17129]: acl.c:748 ast_apply_acl: IAX2 user ACL: Rejecting ‘10.50.0.5’ due to a failure to pass ACL ‘(BASELINE)’
[2014-08-20 13:05:49] NOTICE[17129]: acl.c:748 ast_apply_acl: IAX2 user ACL: Rejecting ‘10.50.0.5’ due to a failure to pass ACL ‘(BASELINE)’
[2014-08-20 13:05:49] NOTICE[17129]: acl.c:748 ast_apply_acl: IAX2 user ACL: Rejecting ‘10.50.0.5’ due to a failure to pass ACL ‘(BASELINE)’
[2014-08-20 13:05:49] NOTICE[17129]: acl.c:748 ast_apply_acl: IAX2 user ACL: Rejecting ‘10.50.0.5’ due to a failure to pass ACL ‘(BASELINE)’
[2014-08-20 13:05:49] NOTICE[17129]: acl.c:748 ast_apply_acl: IAX2 user ACL: Rejecting ‘10.50.0.5’ due to a failure to pass ACL ‘(BASELINE)’
[2014-08-20 13:05:49] NOTICE[17129]: acl.c:748 ast_apply_acl: IAX2 user ACL: Rejecting ‘10.50.0.5’ due to a failure to pass ACL ‘(BASELINE)’

We have search the net for answers, but could not find any definite resolution.
Help will be much appretiated