Skip to content

tests: Move langchain under toxgen, add variants #4349

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 20 commits into
base: master
Choose a base branch
from

Conversation

sentrivana
Copy link
Contributor

@sentrivana sentrivana commented Apr 29, 2025

Move langchain tests under toxgen.

Since there are two variants of langchain tests which should install different dependencies, also add support for this. (We'll need it for other things like openai.)

DB failures are unrelated

Copy link

codecov bot commented Apr 29, 2025

❌ 386 Tests Failed:

Tests completed Failed Passed Skipped
32009 386 31623 8890
View the top 3 failed test(s) by shortest run time
tests.integrations.aiohttp.test_aiohttp test_failed_request_status_codes[integration_kwargs4-HTTPNetworkAuthenticationRequired-True]
Stack Traces | 0.001s run time
file .../integrations/aiohttp/test_aiohttp.py, line 679
  @pytest.mark.parametrize(
      ("integration_kwargs", "exception_to_raise", "should_capture"),
      (
          ({}, None, False),
          ({}, HTTPBadRequest, False),
          (
              {},
              HTTPUnavailableForLegalReasons(None),
              False,
          ),  # Highest 4xx status code (451)
          ({}, HTTPInternalServerError, True),
          ({}, HTTPNetworkAuthenticationRequired, True),  # Highest 5xx status code (511)
          ({"failed_request_status_codes": set()}, HTTPInternalServerError, False),
          (
              {"failed_request_status_codes": set()},
              HTTPNetworkAuthenticationRequired,
              False,
          ),
          ({"failed_request_status_codes": {404, *range(500, 600)}}, HTTPNotFound, True),
          (
              {"failed_request_status_codes": {404, *range(500, 600)}},
              HTTPInternalServerError,
              True,
          ),
          (
              {"failed_request_status_codes": {404, *range(500, 600)}},
              HTTPBadRequest,
              False,
          ),
      ),
  )
  @pytest.mark.asyncio
  async def test_failed_request_status_codes(
      sentry_init,
      aiohttp_client,
      capture_events,
      integration_kwargs,
      exception_to_raise,
      should_capture,
  ):
      sentry_init(integrations=[AioHttpIntegration(**integration_kwargs)])
      events = capture_events()

      async def handle(_):
          if exception_to_raise is not None:
              raise exception_to_raise
          else:
              return web.Response(status=200)

      app = web.Application()
      app.router.add_get("/", handle)

      client = await aiohttp_client(app)
      resp = await client.get("/")

      expected_status = (
          200 if exception_to_raise is None else exception_to_raise.status_code
      )
      assert resp.status == expected_status

      if should_capture:
          (event,) = events
          assert event["exception"]["values"][0]["type"] == exception_to_raise.__name__
      else:
          assert not events
E       fixture 'aiohttp_client' not found
>       available fixtures: DictionaryContaining, ObjectDescribedBy, StringContaining, _capture_internal_warnings, anyio_backend, anyio_backend_name, anyio_backend_options, benchmark, cache, capfd, capfdbinary, caplog, capsys, capsysbinary, capture_envelopes, capture_events, capture_events_forksafe, capture_exceptions, capture_record_lost_event_calls, clean_scopes, cov, doctest_namespace, event_loop, free_tcp_port, free_tcp_port_factory, free_udp_port, free_udp_port_factory, httpserver, httpsserver, internal_exceptions, maybe_monkeypatched_threading, monkeypatch, no_cover, pytestconfig, record_property, record_testsuite_property, record_xml_attribute, recwarn, render_span_tree, reset_integrations, sentry_init, smtpserver, suppress_deprecation_warnings, teardown_profiling, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory, uninstall_integration, validate_event_schema
>       use 'pytest --fixtures [testpath]' for help on them.

.../integrations/aiohttp/test_aiohttp.py:679
tests.integrations.httpx.test_httpx test_crumb_capture_client_error[403-warning-httpx_client0]
Stack Traces | 0.001s run time
file .../integrations/httpx/test_httpx.py, line 60
  @pytest.mark.parametrize(
      "httpx_client",
      (httpx.Client(), httpx.AsyncClient()),
  )
  @pytest.mark.parametrize(
      "status_code,level",
      [
          (200, None),
          (301, None),
          (403, "warning"),
          (405, "warning"),
          (500, "error"),
      ],
  )
  def test_crumb_capture_client_error(
E       fixture 'httpx_mock' not found
>       available fixtures: DictionaryContaining, ObjectDescribedBy, StringContaining, _capture_internal_warnings, anyio_backend, anyio_backend_name, anyio_backend_options, benchmark, cache, capfd, capfdbinary, caplog, capsys, capsysbinary, capture_envelopes, capture_events, capture_events_forksafe, capture_exceptions, capture_record_lost_event_calls, clean_scopes, cov, doctest_namespace, free_tcp_port, free_tcp_port_factory, free_udp_port, free_udp_port_factory, httpserver, httpsserver, internal_exceptions, maybe_monkeypatched_threading, monkeypatch, no_cover, pytestconfig, record_property, record_testsuite_property, record_xml_attribute, recwarn, render_span_tree, reset_integrations, sentry_init, smtpserver, suppress_deprecation_warnings, teardown_profiling, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory, uninstall_integration, validate_event_schema
>       use 'pytest --fixtures [testpath]' for help on them.

.../integrations/httpx/test_httpx.py:60
tests.integrations.httpx.test_httpx test_span_origin[httpx_client1]
Stack Traces | 0.001s run time
file .../integrations/httpx/test_httpx.py, line 395
  @pytest.mark.parametrize(
      "httpx_client",
      (httpx.Client(), httpx.AsyncClient()),
  )
  def test_span_origin(sentry_init, capture_events, httpx_client, httpx_mock):
E       fixture 'httpx_mock' not found
>       available fixtures: DictionaryContaining, ObjectDescribedBy, StringContaining, _capture_internal_warnings, anyio_backend, anyio_backend_name, anyio_backend_options, benchmark, cache, capfd, capfdbinary, caplog, capsys, capsysbinary, capture_envelopes, capture_events, capture_events_forksafe, capture_exceptions, capture_record_lost_event_calls, clean_scopes, cov, doctest_namespace, free_tcp_port, free_tcp_port_factory, free_udp_port, free_udp_port_factory, httpserver, httpsserver, internal_exceptions, maybe_monkeypatched_threading, monkeypatch, no_cover, pytestconfig, record_property, record_testsuite_property, record_xml_attribute, recwarn, render_span_tree, reset_integrations, sentry_init, smtpserver, suppress_deprecation_warnings, teardown_profiling, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory, uninstall_integration, validate_event_schema
>       use 'pytest --fixtures [testpath]' for help on them.

.../integrations/httpx/test_httpx.py:395

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

Base automatically changed from ivana/toxgen/move-more-stuff to master April 29, 2025 09:58
@@ -158,6 +158,7 @@ def test_langchain_agent(
model_name="gpt-3.5-turbo",
temperature=0,
openai_api_key="badkey",
streaming=True,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This prevents a "wrong API key" error on langchain < 1.1

@sentrivana sentrivana marked this pull request as ready for review May 6, 2025 12:26
@sentrivana sentrivana requested a review from a team as a code owner May 6, 2025 12:26
@sentrivana sentrivana changed the title tests: Move langchain under toxgen tests: Move langchain under toxgen, add variants May 6, 2025
@sentrivana sentrivana marked this pull request as draft May 6, 2025 13:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant