High Risk Nonprofit Security Enhancement Act of 2004

High Risk Nonprofit Security Enhancement Act of 2004 - Amends the Homeland Security Act of 2002 to authorize the Secretary of Homeland Security to: (1) enter into contracts for security enhancements and technical assistance for nonprofit organizations; and (2) issue Federal loan guarantees in connection with loans made by financial institutions to nonprofit organizations for security enhancements and technical assistance. Permits the Secretary to guarantee only such loans that have favorable repayment terms.

Directs the Secretary to designate high-risk nonprofit organizations eligible for contracts or loans based on the vulnerability of the specific site of the nonprofit to international terrorist attacks. Sets forth criteria for vulnerability determinations.

Authorizes: (1) the use of federally guaranteed funds borrowed from lending institutions for technical assistance and security enhancements; (2) the Secretary to select applications for execution of security enhancement and technical assistance contracts, or issuance of loan guarantees, giving preference to the nonprofit organizations determined to be at greatest risk of international terrorist attack; (3) each State to establish a State Homeland Security Authority; and (4) the Secretary to provide grants to local governments to offset incremental costs associated with law enforcement in areas with a high concentration of nonprofit organizations.

Establishes within the Department of Homeland Security the Office of Community Relations and Civic Affairs to administer grant programs for nonprofit organizations and local law enforcement assistance.