It's more in how the code has been written. You've set p2 to be the polar of pt at 0 degrees. So if there's no curve along that direction, ssget cannot find any curve and returns nil, thus ssname then yields an error.
I would do it this way:
(defun pt-in-region (pt pts Acc / ss-member-pts pt1 ptn p at ACC AT P PT PT1 PTN PTS)
;;judge a point is in the 2D polygon , polygon given with vetexs .
;;by GSLS(SS) 2011.03.28
;;return 0 at polygon, 1 in it, -1 out it .
(defun ss-member-pts (pt ptl acc / is_go i len a b)
(setq is_go T i 0 len(length ptl))
(while (and is_go (< i len))
(setq a(car ptl)ptl(cdr ptl)i(1+ i))
(if (and a (equal a pt acc))
(setq is_go nil b (cons a ptl)))))
(setq pt1(list (+ (car (apply (function mapcar) (cons (function max) pts)))(abs acc)(abs acc))(cadr pt)))
(mapcar(function(lambda (x y)
(if(setq p (inters pt pt1 x y T))
(progn (if(equal (+(distance pt x)(distance pt y))(distance x y)acc)(setq at T))
(if(not (ss-member-pts p pts acc)) (setq ptn (cons p ptn)))))))
(cons (last pts) pts) pts)
(cond (at 0)
((and (not at) ptn)
(if (= (rem (length ptn) 2) 1)
1 -1 ))
(t -1)
)
)
;;pts pt is
(defun c:test (/ is poly pt pts)
(setq pt (getpoint)
poly (ssget "f" (list pt (polar pt 0 10000)) '((0 . "LWPOLYLINE")))
)
(cond
((not poly) (alert "Out ."))
(t
(setq
pts (mapcar 'cdr
(vl-remove-if-not
'(lambda (x) (= (car x) 10))
(entget (ssname poly 0))
)
)
is (pt-in-region pt pts 1e-8)
)
(cond
((= is -1) (alert "Out ."))
((= is 0) (alert "At ."))
((= is 1) (alert "In ."))
)
)
)
(princ)
)
That being said, your code still yields incorrect results sometimes... That's if there is a horizontal segment along the polyline? I guess it's just up to the function itself: