Apparatus and method for optimizing reliability of satellite system considering both hard error stability and soft error stability