On Wed, Dec 08, 2010 at 06:16:00PM -0800, cmallon wrote:

The subject of your message is misleading and unfortunate. Postfix
behaves exactly as configured.
You only accept mail for domains listed in mydestination,
virtual_alias_domains, virtual_mailbox_domains, or relay_domains.
You have three tables that further filter the recipient domain.
Then a table that filters the sender domain. Anything that makes it past
this rule is accepted.
Then two pointless permit rules that serve no purpose at the end of the
list, since the default is to permit if the end of the list is reached
with no reject.

Thank you Viktor for taking the time out to look at my issue and assist me with. I do appreciate your time and patience.

On Dec 8, 2010, at 9:17 PM, Victor Duchovni wrote:

On Wed, Dec 08, 2010 at 06:16:00PM -0800, cmallon wrote:

The subject of your message is misleading and unfortunate. Postfix
behaves exactly as configured.

smtpd_recipient_restrictions =
reject_unauth_destination,

You only accept mail for domains listed in mydestination,
virtual_alias_domains, virtual_mailbox_domains, or relay_domains.

I'm not sure I understand. Are you saying that my values for mydestination, virtual_alias_domains, virtual_mailbox_domains or relay domains is responsible my broken smtpd_reciepient_restriction? I have 4 postfix servers, this server is dedicated to only one client (my other servers run wonderfully)

check_recipient_access hash:/etc/postfix/recipient_blacklist,
<implicit_check_recipient_access> hash:/etc/postfix/perm_blacklist,
<implicit_check_recipient_access> hash:/etc/postfix/hold,

You have three tables that further filter the recipient domain.

check_sender_access hash:/etc/postfix/sender_access,

Then a table that filters the sender domain. Anything that makes it past
this rule is accepted.

permit_sasl_authenticated, permit_mynetworks

I made some changes and removed check sender access and the hold tables, but the recipient restrictions are still not being honored.

Then two pointless permit rules that serve no purpose at the end of the
list, since the default is to permit if the end of the list is reached
with no reject.
--
Viktor.


Here is the new postconf -n:

mail2:/var/spool/postfix root# postconf -n
2bounce_notice_recipient = postmaster
alias_maps = hash:/etc/aliases
always_bcc =
append_at_myorigin = no
enable_server_options = yes
html_directory = no
inet_interfaces = all
mail_owner = _postfix
mailbox_size_limit = 0
mailbox_transport = cyrus
mailq_path = /usr/bin/mailq
manpage_directory = /usr/share/man
maximal_queue_lifetime = 1d
message_size_limit = 10240000
mydestination = $myhostname,localhost.$mydomain
mydomain = my_clientmail.my_company.com
mydomain_fallback = localhost
myhostname = my_clientmail
mynetworks = 127.0.0.1/32,10.1.0.0/16,192.168.3.0/24,172.16.0.0/12,10.1.18.24,192.168.0.0/16
mynetworks_style = host
newaliases_path = /usr/bin/newaliases
notify_classes = bounce,protocol
owner_request_special = no
queue_directory = /private/var/spool/postfix
readme_directory = /usr/share/doc/postfix
recipient_delimiter = +
sample_directory = /usr/share/doc/postfix/examples
sendmail_path = /usr/sbin/sendmail
setgid_group = _postdrop
smtp_destination_concurrency_limit = 50
smtpd_client_restrictions =
smtpd_enforce_tls = no
smtpd_pw_server_security_options = plain,login,cram-md5,gssapi
smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_blacklist,hash:/etc/postfix/perm_blacklist,permit_sasl_authenticated,permit_mynetworks,reject_unauth_destination
smtpd_sasl_auth_enable = yes
smtpd_tls_key_file =
smtpd_tls_loglevel = 0
smtpd_use_pw_server = yes
smtpd_use_tls = no
unknown_local_recipient_reject_code = 550
virtual_alias_maps = hash:/etc/postfix/virtual_alias
virtual_transport = lmtp:unix:/var/imap/socket/lmtp debug_peer_list = 127.0.0.

I noticed the conversation regarding brevity this morning, so please pardon me if I'm providing too much information but I just want to be clear. I need my mail server to use an internal black list.

I have emails that should never leave my server:

I know this address is in the blacklist table:

mail2:/var/spool/postfix root# grep ***@tiscali.co.uk /etc/postfix/recipient_blacklist
***@tiscali.co.uk reject
mail2:/var/spool/postfix root# grep ***@tiscali.co.uk /etc/postfix/perm_blacklist
***@tiscali.co.uk reject

Yet here it is in my mail logs:

Dec 9 08:59:00 mail2 postfix/smtp[16933]: 0725C8E704FD: to=<***@tiscali.co.uk>, relay=mxgb1.opaltelecom.net[62.24.139.61]:25, delay=14026, delays=14025/0.11/0.67/0.33, dsn=5.0.0, status=bounced (host mxgb1.opaltelecom.net[62.24.139.61] said: 550 #5.1.0 Address rejected ***@tiscali.co.uk (in reply to RCPT TO command))

Here is another that was actually sent:

mail2:/var/spool/postfix root# grep ***@hotmail.com /etc/postfix/recipient_blacklist
***@hotmail.com reject
mail2:/var/spool/postfix root# grep ***@hotmail.com /etc/postfix/perm_blacklist
***@hotmail.com reject

Dec 9 09:21:30 mail2 postfix/smtp[17661]: 0ACFE8E4392A: to=<***@hotmail.com>, relay=127.0.0.1[127.0.0.1]:10027, conn_use=13, delay=23079, delays=23077/1.3/0.01/1.1, dsn=2.0.0, status=sent (250 2.0.0 Ok: queued as 51E0B8ECA5FB)

The current state of my queues:

mail2:/var/spool/postfix root# ls -l
total 24
drwx------ 17 _postfix wheel 578 Dec 9 09:31 active
drwx------ 3 _postfix wheel 102 Dec 9 09:31 bounce
-rwxrwxrwx 1 root wheel 125 Nov 2 16:55 check
drwx------ 2 _postfix wheel 68 Aug 22 2005 corrupt
-rwxrwxrwx 1 root wheel 124 Dec 8 16:01 count
drwx------ 18 _postfix wheel 612 Nov 2 16:36 defer
drwx------ 18 _postfix wheel 612 Mar 16 2007 deferred
drwx------ 3 _postfix wheel 102 Nov 30 07:54 flush
drwx------ 2 _postfix wheel 68 Aug 22 2005 hold
drwx------ 46523 _postfix wheel 50602710 Dec 9 09:31 incoming
drwx------ 57617 _postfix wheel 57664578 Dec 6 12:50 incoming.1206
drwx------ 60089 _postfix wheel 6499474 Dec 6 22:36 incoming.old
-rwxrwxrwx 1 root wheel 572 Dec 8 20:51 mailbyUser
drwx-wx--- 2 _postfix _postdrop 68 Dec 9 08:58 maildrop
drwxr-xr-x 20 root wheel 680 Oct 19 18:10 pid
drwx------ 26 _postfix wheel 884 Dec 8 09:29 private
drwx--x--- 7 _postfix _postdrop 238 Dec 8 08:59 public
drwx------ 2 _postfix wheel 68 Aug 22 2005 saved
drwx------ 17 _postfix wheel 578 Nov 19 2009 trace
mail2:/var/spool/postfix root#

"broken" is in the eye of the beholder. If you want Postfix to behave
differently, you need to configure it differently. I'm just telling
what your configuration means. In this case it rejects all recipient
domains not found in one of the above address classes.
The recipient restrictions are always honored. Unless your master.cf
file overrides main.cf in the "smtpd" instance the client connects
to, what you configure is what you get...
These IPs will pass "permit_mynetworks".
A bit too aggressive IMHO, many sites will not tolerate this, and you
just reduce performance.
An Apple customization, we don't support this here.
Recipients in this table are handled as specified.
If not rejected or definitely accepted by the previous rule,
handled as specified in this table.
If not already rejected or accepted, trusted or authenticated
clients can send to anyone.
Otherwise only domains in the usual address classes are accepted.
Apple-specific.
Don't "grep", use "postmap -q".
How did this message enter your system? Where the rest of the logging
for the queue-id in question? What was the state of "main.cf" at the
time.
Don't "grep", rather "postmap -q".
Your content filter is sure slow, you have a 6+ hour delay in your
internal filter, something is very wrong.
This is really bad. Your incoming queues are huge. And you are manually
renaming directories in the queue to try to fix it, this is no way to
run a Postfix server...

http://www.postfix.org/QSHAPE_README.html

On Dec 9, 2010, at 10:55 AM, Victor Duchovni wrote:


The recipient restrictions are always honored. Unless your master.cf
file overrides main.cf in the "smtpd" instance the client connects
to, what you configure is what you get...

This is the master.cf file:
# DO NOT SHARE THE POSTFIX QUEUE BETWEEN MULTIPLE POSTFIX INSTANCES.
#
# ==========================================================================
# service type private unpriv chroot wakeup maxproc command + args
# (yes) (yes) (yes) (never) (100)
# ==========================================================================
smtp inet n - n - - smtpd
-o smtpd_sasl_auth_enable=yes
#
# Before-filter SMTP server. Receive mail from the network and
# pass it to the content filter on localhost port 10025.
#
#smtp inet n - n - - smtpd
# -o smtpd_proxy_filter=127.0.0.1:10025
# -o smtpd_client_connection_count_limit=10
#
# After-filter SMTP server. Receive mail from the content filter on
# localhost port 10026.
#
#127.0.0.1:10026 inet n - n - - smtpd
# -o smtpd_authorized_xforward_hosts=127.0.0.0/8
# -o smtpd_client_restrictions=
# -o smtpd_helo_restrictions=
# -o smtpd_sender_restrictions=
# -o smtpd_recipient_restrictions=permit_mynetworks,reject
# -o smtpd_data_restrictions=
# -o mynetworks=127.0.0.0/8
# -o receive_override_options=no_unknown_recipient_checks
#
submission inet n - n - - smtpd
-o smtpd_etrn_restrictions=reject
-o smtpd_sasl_auth_enable=yes
-o content_filter=dksign:[127.0.0.1]:10027
-o receive_override_options=no_address_mappings
-o smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_blacklist,hash:/etc/postfix/perm_blacklist,permit_mynetworks,reject
#
#
# specify the location of the DomainKeys signing filter
#
dksign unix - - n - 10 smtp
-o smtp_send_xforward_command=yes
-o smtp_discard_ehlo_keywords=8bitmime
#
#
# service for accepting messages FROM the DomainKeys signing filter
#
#smtp inet n - n - - smtpd
127.0.0.1:10028 inet n - n - 10 smtpd
-o content_filter=
-o receive_override_options=no_unknown_recipient_checks,no_header_body_checks
-o smtpd_helo_restrictions=
-o smtpd_client_restrictions=
-o smtpd_sender_restrictions=
-o smtpd_recipient_restrictions=permit_mynetworks,reject
-o mynetworks=127.0.0.0/8
-o smtpd_authorized_xforward_hosts=127.0.0.0/8
#
#smtps inet n - n - - smtpd
# -o smtpd_tls_wrappermode=yes -o smtpd_sasl_auth_enable=yes
#submission inet n - n - - smtpd
# -o smtpd_etrn_restrictions=reject
# -o smtpd_enforce_tls=yes -o smtpd_sasl_auth_enable=yes
#628 inet n - n - - qmqpd
pickup fifo n - n 60 1 pickup
-o content_filter=
cleanup unix n - n - 0 cleanup
qmgr fifo n - n 300 1 qmgr
#qmgr fifo n - n - 1 nqmgr
#tlsmgr fifo - - n - 1 tlsmgr
rewrite unix - - n - - trivial-rewrite
bounce unix - - n - 0 bounce
defer unix - - n - 0 bounce
trace unix - - n - 0 bounce
verify unix - - n - 1 verify
flush unix n - n 1000? 0 flush
proxymap unix - - n - - proxymap
smtp unix - - n - - smtp
relay unix - - n - - smtp
# -o smtp_helo_timeout=5 -o smtp_connect_timeout=5
showq unix n - n - - showq
error unix - - n - - error
local unix - n n - - local
virtual unix - n n - - virtual
lmtp unix - - n - - lmtp
anvil unix - - n - 1 anvil
#
#
# Interfaces to non-Postfix software. Be sure to examine the manual
# pages of the non-Postfix software to find out what options it wants.
#
# maildrop. See the Postfix MAILDROP_README file for details.
#
maildrop unix - n n - - pipe
flags=DRhu user=vmail argv=/usr/local/bin/maildrop -d ${recipient}
#
# The Cyrus deliver program has changed incompatibly, multiple times.
#
old-cyrus unix - n n - - pipe
flags=R user=cyrusimap argv=/cyrus/bin/deliver -e -m ${extension} ${user}
# Cyrus 2.1.5 (Amos Gouaux)
# Also specify in main.cf: cyrus_destination_recipient_limit=1
cyrus unix - n n - 10 pipe
user=cyrusimap argv=/usr/bin/cyrus/bin/deliver -e -r ${sender} -m ${extension} ${user}
uucp unix - n n - - pipe
flags=Fqhu user=uucp argv=uux -r -n -z -a$sender - $nexthop!rmail ($recipient)
ifmail unix - n n - - pipe
flags=F user=ftn argv=/usr/lib/ifmail/ifmail -r $nexthop ($recipient)
bsmtp unix - n n - - pipe
flags=Fq. user=foo argv=/usr/local/sbin/bsmtp -f $sender $nexthop $recipient
scache unix - - n - 1 scache
discard unix - - n - - discard
tlsmgr unix - - n 1000? 1 tlsmgr
retry unix - - n - - error



These IPs will pass "permit_mynetworks".

smtp_destination_concurrency_limit = 50

A bit too aggressive IMHO, many sites will not tolerate this, and you
just reduce performance.

REMOVED



smtpd_recipient_restrictions =
check_recipient_access hash:/etc/postfix/recipient_blacklist,

Recipients in this table are handled as specified.

That's my problem -- they aren't. All the recipient addresses in this table are marked "reject"

If not already rejected or accepted, trusted or authenticated
clients can send to anyone.

reject_unauth_destination

Otherwise only domains in the usual address classes are accepted.


I know this address is in the blacklist table:

mail2:/var/spool/postfix root# grep ***@tiscali.co.uk /etc/postfix/recipient_blacklist
***@tiscali.co.uk reject

Don't "grep", use "postmap -q".

mail2:/var/spool/postfix root# postmap -q ***@tiscali.co.uk /etc/postfix/recipient_blacklist
reject


How did this message enter your system?

The client's customer signs up to receive notifications on a topic they're interested in. When the topic is updated, the application on our server generates an email. The email is sent to a vip on our load balancer then sent to the internal client mail server. (This was set up with lad balancing in mind). At this time, the mail server should sign the email using dkfilter and send the email off to the subscriber. Sometimes, someone will fat finger their address, have a name change or discontinue service with their isp and fail to unsubscribe from the notification list. Thats when we get a bounce. I'd like to prevent sending emails to bad addresses, so I set up an internal blacklist. Maybe smtpd_recipient_restrictions isn't the correct parameter for this?

Where the rest of the logging
for the queue-id in question? What was the state of "main.cf" at the
time.


Dec 9 08:58:59 mail2 postfix/qmgr[16878]: 0725C8E704FD: from=<***@myclient.com>, size=2019, nrcpt=1 (queue active)
Dec 9 08:59:00 mail2 postfix/smtp[16933]: 0725C8E704FD: to=<***@tiscali.co.uk>, relay=mxgb1.opaltelecom.net[62.24.139.61]:25, delay=14026, delays=14025/0.11/0.67/0.33, dsn=5.0.0, status=bounced (host mxgb1.opaltelecom.net[62.24.139.61] said: 550 #5.1.0 Address rejected ***@tiscali.co.uk (in reply to RCPT TO command))
Dec 9 08:59:00 mail2 postfix/bounce[16967]: 0725C8E704FD: sender non-delivery notification: 7D1578EC1C27
Dec 9 08:59:01 mail2 postfix/bounce[16967]: 0725C8E704FD: postmaster non-delivery notification: 857938EC1C42
Dec 9 08:59:01 mail2 postfix/qmgr[16878]: 0725C8E704FD: removed

The state of the main.cf:
2bounce_notice_recipient = postmaster
alias_maps = hash:/etc/aliases
always_bcc =
append_at_myorigin = no
enable_server_options = yes
html_directory = no
inet_interfaces = all
mail_owner = _postfix
mailbox_size_limit = 0
mailbox_transport = cyrus
mailq_path = /usr/bin/mailq
manpage_directory = /usr/share/man
maximal_queue_lifetime = 1d
message_size_limit = 10240000
mydestination = $myhostname,localhost.$mydomain
mydomain = my_clientmail.my_company.com
mydomain_fallback = localhost
myhostname = my_clientmail
mynetworks = 127.0.0.1/32,10.1.0.0/16,192.168.3.0/24,172.16.0.0/12,10.1.18.24,192.168.0.0/16
mynetworks_style = host
newaliases_path = /usr/bin/newaliases
notify_classes = bounce,protocol
owner_request_special = no
queue_directory = /private/var/spool/postfix
readme_directory = /usr/share/doc/postfix
recipient_delimiter = +
sample_directory = /usr/share/doc/postfix/examples
sendmail_path = /usr/sbin/sendmail
setgid_group = _postdrop
smtp_destination_concurrency_limit = 50
smtpd_client_restrictions =
smtpd_enforce_tls = no
smtpd_pw_server_security_options = plain,login,cram-md5,gssapi
smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_blacklist,hash:/etc/postfix/perm_blacklist,permit_sasl_authenticated,permit_mynetworks,reject_unauth_destination
smtpd_sasl_auth_enable = yes
smtpd_tls_key_file =
smtpd_tls_loglevel = 0
smtpd_use_pw_server = yes
smtpd_use_tls = no
unknown_local_recipient_reject_code = 550
virtual_alias_maps = hash:/etc/postfix/virtual_alias
virtual_transport = lmtp:unix:/var/imap/socket/lmtp debug_peer_list = 127.0.0.



Your content filter is sure slow, you have a 6+ hour delay in your
internal filter, something is very wrong.

Yes -- I know -- and I recognize that I may have two separate problems going on.


drwx------ 46523 _postfix wheel 50602710 Dec 9 09:31 incoming
drwx------ 57617 _postfix wheel 57664578 Dec 6 12:50 incoming.1206
drwx------ 60089 _postfix wheel 6499474 Dec 6 22:36 incoming.old

This is really bad. Your incoming queues are huge. And you are manually
renaming directories in the queue to try to fix it, this is no way to
run a Postfix server...

Your right. I've been trying to solve this all week and this is why I'm coming to you...

http://www.postfix.org/QSHAPE_README.html

I was under the impression that QSHAPE wouldn't run on MAC OS -- but I'll give it a shot and get back to you with the results.

The message your reported as 'delivered' before came in on the submission
"port 587" port, as evidenced by the huge queue for dkim signing.

You have botched the master.cf entry for this, since white-space is
not allowed in "-o parameter=value" master.cf overrides. I'd expect
to see warnings in your logs to that effect.
The master.cf file overrides this (incorrectly) for the submission service.
That's better.
I mean logs!!!
Useless! Where's the smtpd or pickup logging.
The qshape Perl code is reasonably portable.

n Dec 9, 2010, at 12:48 PM, Victor Duchovni wrote:

On Thu, Dec 09, 2010 at 03:31:21PM -0500, cmallon wrote:

submission inet n - n - - smtpd
-o smtpd_etrn_restrictions=reject
-o smtpd_sasl_auth_enable=yes
-o content_filter=dksign:[127.0.0.1]:10027

-o receive_override_options=no_address_mappings
-o smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_blacklist,hash:/etc/postfix/perm_blacklist,permit_mynetworks,reject

The message your reported as 'delivered' before came in on the submission
"port 587" port, as evidenced by the huge queue for dkim signing.

Is it possible that there is too much mail and dkim can't process it fast enough?

You have botched the master.cf entry for this, since white-space is
not allowed in "-o parameter=value" master.cf overrides. I'd expect
to see warnings in your logs to that effect.

smtpd_recipient_restrictions =
check_recipient_access hash:/etc/postfix/recipient_blacklist,

Recipients in this table are handled as specified.

That's my problem -- they aren't. All the recipient addresses in this
table are marked "reject"

The master.cf file overrides this (incorrectly) for the submission service.

Can I safely remove this parameter from the master.cf file? (smtpd_recipient_restrictions); would that solve my problem and simplify my configuration? Or do I need this parameter in this location because of the way mail is passed through the domain keys?

That depends on what restrictions you want/need to apply to port 587
vs. port 25. In

http://www.postfix.org/master.5.html

under the description of "-o" arguments, you'll find how to correctly
specify complex parameter overrides in the master.cf file.

-o name=value
Override the named main.cf configuration
parameter. The parameter value can refer to
other parameters as $name etc., just like in
main.cf. See postconf(5) for syntax.

NOTE 1: do not specify whitespace around the
"=". In parameter values, either avoid
whitespace altogether, use commas instead of
spaces, or consider overrides like "-o
name=$override_parameter" with $over-
ride_parameter set in main.cf.

NOTE 2: Over-zealous use of parameter over-
rides makes the Postfix configuration hard
to understand and maintain. At a certain
point, it might be easier to configure mul-
tiple instances of Postfix, instead of con-
figuring multiple personalities via mas-
ter.cf.

I'm reluctant to be this blunt, but you need to learn more about running
a Postfix (or some other) MTA before you take on customer traffic. There
are multiple serious issues with your MTA and you're in over your head.

n Dec 9, 2010, at 1:59 PM, Victor Duchovni wrote:

On Thu, Dec 09, 2010 at 01:45:37PM -0800, cmallon wrote:

Can I safely remove this parameter from the master.cf file?
(smtpd_recipient_restrictions); would that solve my problem and simplify
my configuration? Or do I need this parameter in this location because
of the way mail is passed through the domain keys?

That depends on what restrictions you want/need to apply to port 587
vs. port 25. In

All mail on this server must pass through domain keys which is through port 587. I've successfully run this server for this client for the last 3 years. We recently upgraded the OS and the client started sending a lot more mail (30,000 pieces at a time)- That's when we started to run into some problems. I might add that I'm not the only admin on site.

http://www.postfix.org/master.5.html

under the description of "-o" arguments, you'll find how to correctly
specify complex parameter overrides in the master.cf file.

-o name=value
Override the named main.cf configuration
parameter. The parameter value can refer to
other parameters as $name etc., just like in
main.cf. See postconf(5) for syntax.

NOTE 1: do not specify whitespace around the
"=". In parameter values, either avoid
whitespace altogether, use commas instead of
spaces, or consider overrides like "-o
name=$override_parameter" with $over-
ride_parameter set in main.cf.

I did remove the whitespaces around the "=" and even simplified the smtpd_recipient_restrictions value in the master.cf but blacklisted mail still gets through.

NOTE 2: Over-zealous use of parameter over-
rides makes the Postfix configuration hard
to understand and maintain. At a certain
point, it might be easier to configure mul-
tiple instances of Postfix, instead of con-
figuring multiple personalities via mas-
ter.cf.
I will go over this document and see if configuring main.cf to handle smtpd_recipient_restrictions is appropriate.

I'm reluctant to be this blunt, but you need to learn more about running
a Postfix (or some other) MTA before you take on customer traffic. There
are multiple serious issues with your MTA and you're in over your head.

Thank you for your bluntness Viktor. You are right, I don't play around with the mail service once it's running but I will endeavor to "learn more about running a Postfix MTA" however, the situation is here and I need to address it. Personally I think it's the domain keys..I don't think they can keep up with the volume. I do thank you for your time and assistance though, you've been very generous and I do appreciate it.