Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Multiple TestCli failures on openSUSE Tumbleweed #2787

Open
glaubitz opened this issue Jul 4, 2023 · 2 comments
Open

Multiple TestCli failures on openSUSE Tumbleweed #2787

glaubitz opened this issue Jul 4, 2023 · 2 comments

Comments

@glaubitz
Copy link

glaubitz commented Jul 4, 2023

CloudFormation Lint Version

0.77.10

What operating system are you using?

openSUSE Tumbleweed

Describe the bug

The testsuite of cfn-lint 0.77.10 fails on openSUSE Tumbleweed with the following four failures:

[   57s] =================================== FAILURES ===================================
[   57s] ___________________________ TestCli.test_bad_config ____________________________
[   57s] 
[   57s] self = <test.unit.module.core.test_run_cli.TestCli testMethod=test_bad_config>
[   57s] yaml_mock = <MagicMock name='_read_config' id='140666292499504'>
[   57s] 
[   57s]     @patch("cfnlint.config.ConfigFileArgs._read_config", create=True)
[   57s]     def test_bad_config(self, yaml_mock):
[   57s]         """Test bad formatting in config"""
[   57s]         yaml_mock.side_effect = [{}, {}]
[   57s]     
[   57s]         filename = "test/fixtures/templates/bad/core/config_parameters.yaml"
[   57s]         (args, _, _) = cfnlint.core.get_args_filenames(
[   57s]             ["--template", filename, "--ignore-bad-template"]
[   57s]         )
[   57s]     
[   57s]         self.assertEqual(args.append_rules, [])
[   57s]         self.assertEqual(args.format, None)
[   57s]         self.assertEqual(args.ignore_bad_template, True)
[   57s]         self.assertEqual(args.ignore_checks, [])
[   57s]         self.assertEqual(args.include_checks, ["W", "E"])
[   57s]         self.assertEqual(args.listrules, False)
[   57s]         self.assertEqual(args.debug, False)
[   57s]         self.assertEqual(args.override_spec, None)
[   57s]         self.assertEqual(args.custom_rules, None)
[   57s] >       self.assertEqual(args.regions, ["us-east-1"])
[   57s] E       AssertionError: Lists differ: ['af-south-1'] != ['us-east-1']
[   57s] E       
[   57s] E       First differing element 0:
[   57s] E       'af-south-1'
[   57s] E       'us-east-1'
[   57s] E       
[   57s] E       - ['af-south-1']
[   57s] E       + ['us-east-1']
[   57s] 
[   57s] test/unit/module/core/test_run_cli.py:244: AssertionError
[   57s] _______________________ TestCli.test_override_parameters _______________________
[   57s] 
[   57s] self = <test.unit.module.core.test_run_cli.TestCli testMethod=test_override_parameters>
[   57s] yaml_mock = <MagicMock name='_read_config' id='140666283394288'>
[   57s] 
[   57s]     @patch("cfnlint.config.ConfigFileArgs._read_config", create=True)
[   57s]     def test_override_parameters(self, yaml_mock):
[   57s]         """Test overriding parameters"""
[   57s]         yaml_mock.side_effect = [{}, {}]
[   57s]     
[   57s]         filename = "test/fixtures/templates/good/core/config_parameters.yaml"
[   57s]         (args, _, _) = cfnlint.core.get_args_filenames(
[   57s]             [
[   57s]                 "--template",
[   57s]                 filename,
[   57s]                 "--ignore-bad-template",
[   57s]                 "--ignore-checks",
[   57s]                 "E0000",
[   57s]             ]
[   57s]         )
[   57s]     
[   57s]         self.assertEqual(args.append_rules, [])
[   57s]         self.assertEqual(args.format, None)
[   57s]         self.assertEqual(args.ignore_bad_template, True)
[   57s]         self.assertEqual(args.ignore_checks, ["E0000"])
[   57s]         self.assertEqual(args.include_checks, ["W", "E"])
[   57s]         self.assertEqual(args.listrules, False)
[   57s]         self.assertEqual(args.debug, False)
[   57s]         self.assertEqual(args.override_spec, None)
[   57s]         self.assertEqual(args.custom_rules, None)
[   57s] >       self.assertEqual(args.regions, ["us-east-1"])
[   57s] E       AssertionError: Lists differ: ['af-south-1'] != ['us-east-1']
[   57s] E       
[   57s] E       First differing element 0:
[   57s] E       'af-south-1'
[   57s] E       'us-east-1'
[   57s] E       
[   57s] E       - ['af-south-1']
[   57s] E       + ['us-east-1']
[   57s] 
[   57s] test/unit/module/core/test_run_cli.py:218: AssertionError
[   57s] _________________ TestCli.test_positional_template_parameters __________________
[   57s] 
[   57s] self = <test.unit.module.core.test_run_cli.TestCli testMethod=test_positional_template_parameters>
[   57s] yaml_mock = <MagicMock name='_read_config' id='140666295874272'>
[   57s] 
[   57s]     @patch("cfnlint.config.ConfigFileArgs._read_config", create=True)
[   57s]     def test_positional_template_parameters(self, yaml_mock):
[   57s]         """Test overriding parameters"""
[   57s]         yaml_mock.side_effect = [{}, {}]
[   57s]     
[   57s]         filename = "test/fixtures/templates/good/core/config_parameters.yaml"
[   57s]         (args, _, _) = cfnlint.core.get_args_filenames(
[   57s]             [filename, "--ignore-bad-template", "--ignore-checks", "E0000"]
[   57s]         )
[   57s]     
[   57s]         self.assertEqual(args.append_rules, [])
[   57s]         self.assertEqual(args.format, None)
[   57s]         self.assertEqual(args.ignore_bad_template, True)
[   57s]         self.assertEqual(args.ignore_checks, ["E0000"])
[   57s]         self.assertEqual(args.include_checks, ["W", "E"])
[   57s]         self.assertEqual(args.listrules, False)
[   57s]         self.assertEqual(args.debug, False)
[   57s]         self.assertEqual(args.override_spec, None)
[   57s]         self.assertEqual(args.custom_rules, None)
[   57s] >       self.assertEqual(args.regions, ["us-east-1"])
[   57s] E       AssertionError: Lists differ: ['af-south-1'] != ['us-east-1']
[   57s] E       
[   57s] E       First differing element 0:
[   57s] E       'af-south-1'
[   57s] E       'us-east-1'
[   57s] E       
[   57s] E       - ['af-south-1']
[   57s] E       + ['us-east-1']
[   57s] 
[   57s] test/unit/module/core/test_run_cli.py:186: AssertionError
[   57s] _________________________ TestCli.test_template_config _________________________
[   57s] 
[   57s] self = <test.unit.module.core.test_run_cli.TestCli testMethod=test_template_config>
[   57s] yaml_mock = <MagicMock name='_read_config' id='140666292413152'>
[   57s] 
[   57s]     @patch("cfnlint.config.ConfigFileArgs._read_config", create=True)
[   57s]     def test_template_config(self, yaml_mock):
[   57s]         """Test template config"""
[   57s]         yaml_mock.side_effect = [{}, {}]
[   57s]     
[   57s]         filename = "test/fixtures/templates/good/core/config_parameters.yaml"
[   57s]         (
[   57s]             args,
[   57s]             _,
[   57s]             _,
[   57s]         ) = cfnlint.core.get_args_filenames(
[   57s]             ["--template", filename, "--ignore-bad-template"]
[   57s]         )
[   57s]     
[   57s]         self.assertEqual(args.append_rules, [])
[   57s]         self.assertEqual(args.append_rules, [])
[   57s]         self.assertEqual(args.format, None)
[   57s]         self.assertEqual(args.ignore_bad_template, True)
[   57s]         self.assertEqual(args.ignore_checks, [])
[   57s]         self.assertEqual(args.include_checks, ["W", "E"])
[   57s]         self.assertEqual(args.listrules, False)
[   57s]         self.assertEqual(args.debug, False)
[   57s]         self.assertEqual(args.override_spec, None)
[   57s]         self.assertEqual(args.custom_rules, None)
[   57s] >       self.assertEqual(args.regions, ["us-east-1"])
[   57s] E       AssertionError: Lists differ: ['af-south-1'] != ['us-east-1']
[   57s] E       
[   57s] E       First differing element 0:
[   57s] E       'af-south-1'
[   57s] E       'us-east-1'
[   57s] E       
[   57s] E       - ['af-south-1']
[   57s] E       + ['us-east-1']
[   57s] 
[   57s] test/unit/module/core/test_run_cli.py:147: AssertionError
[   57s] =========================== short test summary info ============================
[   57s] FAILED test/unit/module/core/test_run_cli.py::TestCli::test_bad_config - Asse...
[   57s] FAILED test/unit/module/core/test_run_cli.py::TestCli::test_override_parameters
[   57s] FAILED test/unit/module/core/test_run_cli.py::TestCli::test_positional_template_parameters
[   57s] FAILED test/unit/module/core/test_run_cli.py::TestCli::test_template_config
[   57s] ================= 4 failed, 591 passed, 2 deselected in 47.98s =================

Expected behavior

Testsuite should complete without failures.

Reproduction template

(n/a)
@kddejong
Copy link
Contributor

kddejong commented Jul 5, 2023

How are you running those tests?

@glaubitz
Copy link
Author

glaubitz commented Jul 6, 2023

How are you running those tests?

Test are run from within the RPM spec file using pytest:

%check
export LANG=en_US.UTF-8
export AWS_DEFAULT_REGION=us-east-1
# the code calls git grep, need to be inside git repo
git init
git add src/cfnlint/rules
# deselect online tests
donttest="test_sarif_formatter"
donttest="$donttest or test_update_resource_specs_python_2"
donttest="$donttest or test_update_resource_specs_python_3"
%pytest -s test -v -k "not ($donttest)"

See: https://build.opensuse.org/package/view_file/home:glaubitz:branches:devel:languages:python:aws/python-cfn-lint/python-cfn-lint.spec?expand=1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@glaubitz @kddejong