The paper was to reduce biased estimation using new approach (Penalized MAXIMUM LIKELIHOOD Estimation (PMLE) Method) in Logistic Regression. For this aim, unreal four small data sets were randomly generated. MAXIMUM LIKELIHOOD Estimation (MLE) and PMLE Methods were applied and compared for separation case including biased estimation in Logistic Regression when one of the cells in 2 x 2 tables becomes equal to zero (separation problem). Parameters1 and their standard error obtained by using MLE for four data sets were 12.56±257.8, 13.46±264.3, 13.42±210.3, and 13.41±180.4, respectively, meaning that MLE’s are biased estimates. Corresponding values for PMLE method were found 2.28 ± 1.81, 3.05 ± 1.59, 3.45 ± 1.53, and 3.45 ± 1.53, respectively, meaning that PMLE’s was unbiased estimates. It is clear that standard error value for data set 1 reduced from 257.8 to 1.81 when using PMLE method for separation problem. According to PMLE Method, the odds of being coronary heart disease risk for smokers were increased 21.08 times than that for non-smokers smoking in data set 2, which is significant at 1% level. The odds of being coronary heart disease risk for smokers were increased 31.63 times than that for non-smokers in data set 3 (P < 0.001). The odds of being coronary heart disease risk for smokers were increased 41.93 times than that for non-smokers in data set 4. When one of the cells in 2 x 2 contingency tables becomes equal to zero, PMLE was more superior to MLE Method because PMLE Method may be performed unbiased (reliable) estimation.